feat(async-publisher): degrade AsyncPubSubPublisher to noop when GCP credentials are absent - #411
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
4fc4ba4 to
0bdb429
Compare
|
I'd be inclined to roll this into #388 instead of keeping it as separate. The changes here lift the detection of GCP credentials up a layer, which is probably the right way forwards. You could then have a metric which is "configuration.errors" which would probably more clearly communicate this issue than at publish time, since it's only checked on startup, and credentials being revoked as the worker is running won't drop into a disabled state, they'll just show up as a failure spike |
…CP creds Extract the cached, lock-guarded GCP credential check out of the sync PubSubPublisher into a shared osprey.worker.lib.gcp_credentials helper, and apply the same noop-when-credentials-absent behavior to the async worker's AsyncPubSubPublisher: skip building the client, log a one-time warning, and short-circuit the publish and stop paths with an async_pubsub_publisher.publish.noop metric for dashboard visibility. Both publishers now share one credential-detection implementation, so a future change (e.g. bounding the metadata probe with a timeout) lands in one place. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TEcfpqhza3dPepZUVWw36X
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TEcfpqhza3dPepZUVWw36X
The shared credential check moved to gcp_credentials_available; update the validation-exporter conftest patch to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TEcfpqhza3dPepZUVWw36X
Mirror the sync-publisher review changes on AsyncPubSubPublisher: drop the per-send async_pubsub_publisher.publish.noop metric in favor of the one-time startup warning, and honor the shared DISABLE_GCP_PUBSUB env opt-out that short-circuits before the credential probe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TEcfpqhza3dPepZUVWw36X
0bdb429 to
48cefda
Compare
48cefda
into
julietshen/noop-publisher-fallback
|
(combining this with #388) |
Second piece of #410 (part of #343). Stacked on #388 — base this on
julietshen/noop-publisher-fallback; review after #388 lands.What this changes
Applies the noop-when-GCP-credentials-absent behavior from #388 to the async worker's
AsyncPubSubPublisher, and dedups the credential check so both publishers share one implementation.osprey/worker/lib/gcp_credentials.py—gcp_credentials_available(), the cached, lock-guarded probe ofgoogle.auth.default()(at most one call per process). Lifted out of the syncPubSubPublisherso there is a single source of truth. A future change (e.g. bounding the metadata probe with a timeout) now lands in one place.publisher.py— drops its private_check_gcp_credentialscopy and imports the shared helper. Credential-check tests move totest_gcp_credentials.py; the publisher tests now patch the shared helper directly.AsyncPubSubPublisher— detects missing credentials at construction, skips building thePublisherClient, logs a one-time warning naming the inert project and topic, and short-circuitspublish_bytes()(both publish entry points route through it) andstop(). Emitsasync_pubsub_publisher.publish.noopper attempt, tagged with project and topic, for parity with the sync side.Verification
ruffclean across all changed files;mypyclean on both publishers and the shared helper.lib/testsare blocked locally by the postgres autouse fixture (same as feat(publisher): degrade PubSubPublisher to noop when GCP credentials are absent #388); verified the sync refactor standalone (shared-helper false/cached paths, sync noop publish/stop, enabled path). CI runs the full suite.Not in this PR
Broader GCP-eager modules (
secret_manager.py,bigquery.py,access_audit_log.py) remain tracked under #343.🤖 Generated with Claude Code
https://claude.ai/code/session_01TEcfpqhza3dPepZUVWw36X