feat(ci): add unit-tests gate to build-and-publish (DISTR-456)#1774
Merged
Conversation
Block publish unless the consumer repo's Tests workflow has reported a successful `unit-tests` check (or whatever name is passed via `unit_tests_check_name`) for the SHA being published. The gate runs before any build minutes are spent so failing tests or missing-coverage checks surface at minute 0 rather than after the multi-arch build. Behaviour: - Check absent on SHA -> fail with onboarding instructions - Check still running -> poll every 30s, max 30 min - Check failed -> fail with the run's conclusion - Check succeeded -> proceed to prepare / build / publish Only runs when `publish: true`; build-only invocations are unaffected. Adds `checks: read` to the workflow permissions for the Checks API lookups. The wait action is pinned to its v1.2.0 SHA per the supply-chain rules in CLAUDE.md. Contract and reference template are documented in docs/standards/unit-tests-gate.md.
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
Contributor
📜 Docstring Coverage ReportRESULT: PASSED (minimum: 30.0%, actual: 77.9%) Detailed Coverage Report |
Contributor
📦 Trivy Vulnerability Scan Results
Report SummaryCould not generate summary table (data length mismatch: 9 vs 8). Scan Result Detailsrequirements.txtuv.lock |
Contributor
📦 Trivy Secret Scan Results
Report SummaryCould not generate summary table (data length mismatch: 9 vs 8). Scan Result Detailsrequirements.txtuv.lock |
Collaborator
☂️ Python Coverage
Overall Coverage
New FilesNo new covered files... Modified FilesNo covered modified files...
|
Contributor
|
🛠 Full Test Coverage Report: https://k.atlan.dev/coverage/application-sdk/pr/1774 |
Collaborator
🧪 SDR Integration Tests (testcontainer) — mysqlStatus: Passed ✅ Temporal workflows: ✅ workflow_include_main_db · ✅ workflow_mixed_filters Extracted + transformed assetsWorkflow run 1 — mysql-0822058c0bd5691a-70d3ed16
Workflow run 2 — mysql-3b377dc2ca6be340-54a68a93
What this validates
Re-runPush a new commit, or re-run from the Actions tab. |
…tion The pre-flight check in unit-tests-gate was using the default per_page=30 on the /check-runs endpoint and a client-side jq filter. On commits with many Dependabot or other bot-created check runs, the target unit-tests check can fall to page 2+ and be invisible — causing the gate to abort with a spurious "check missing" error even when the check exists and passed. Switch to the API's server-side `check_name` query parameter, which filters before paginating and exposes the count directly via .total_count. Discovered while testing on atlan-clickhouse-app, where the SHA on main had 31 check-runs (15 of them Dependabot) and unit-tests sat at position 31.
…ck found Adds an optional `unit_tests_workflow_file` input. When set, the gate's pre-flight no longer fails on a missing check — instead it dispatches the named workflow on the publish ref and falls through to the wait step, which then picks up the newly-created check once GitHub registers the run. Mode summary: - Polling-only (default, when input unset): existing behaviour — gate fails fast if no check on SHA. Lower latency, requires the app's tests workflow to run on every publish commit independently (e.g. via push:main). - Hybrid (when input set): freshness on demand. Requires the target workflow to declare workflow_dispatch:. Limitation: gh workflow run accepts a branch or tag, never a bare SHA, so the dispatched run executes on the branch's current HEAD. For typical publish flows this matches the publish SHA exactly; edge cases (commit moves between resolve and dispatch) accept millisecond-scale drift.
Make `unit_tests_workflow_file` required and remove `unit_tests_check_name`. The gate now tracks tests by run ID end-to-end: - Look for an existing run of the named workflow on the publish SHA. - If none, dispatch on the publish ref and capture the new run's ID. - Poll that run by ID until completion; require conclusion=success. This removes the polling-only / hybrid mode split and the check-name plumbing. Apps no longer need to expose a check called `unit-tests` — the gate works regardless of how the test job is named. Tests workflow only needs to declare `workflow_dispatch:` and exit non-zero on failure. Replaces `fountainhead/action-wait-for-check` with an inline `gh run view` poll loop, dropping a third-party SHA pin in the process.
anurag-atlan
requested review from
Aryamanz29,
OnkarVO7,
atlan-ci,
cmgrote,
louisnow and
vaibhavatlan
as code owners
Apps that haven't onboarded to the gate yet keep working — the gate skips itself when the input is unset. Enforcement comes later via a one-line follow-up that flips required: false → true. Reasoning: the 72 consumer draft PRs are still draft (waiting for this SDK PR to land). Making the input required would break every existing caller the moment this merges, before consumer PRs can be merged in sequence. This change lets the SDK PR land safely, then consumers opt in at their pace.
anurag-atlan
added a commit
that referenced
this pull request
May 19, 2026
…-456 follow-up) Flip the gate from opt-in (non-breaking) to mandatory. Every caller of build-and-publish-app.yaml MUST now set unit_tests_workflow_file; callers that don't are rejected by workflow_call at startup. This is the enforcement-flip follow-up to #1774. Sequence: 1. #1774 lands the gate as opt-in (zero breakage to existing callers). 2. The 72 consumer draft PRs are merged at each app team's pace, each setting unit_tests_workflow_file on its caller. 3. THIS PR flips the input to required, forcing any laggard apps to onboard before they can publish again. Do NOT merge this PR until consumer adoption is high enough. Any app that has not merged its consumer PR by the time this lands will fail to publish until they do.
While the gate is opt-in (unit_tests_workflow_file optional), make gate failure a warning rather than a hard block. Lets app teams merge their consumer PRs even when their current tests are broken — they get a visible warning but their publishes continue working. Once the enforcement-flip PR (#1788) lands, this reverts to exit 1 and gate failure blocks publish properly. Combined with workflow_file being optional, this gives a 2-step ramp: 1. Today: opt-in + warn-only (no breakage possible) 2. Enforcement flip: required + hard-block (laggards get forced)
OnkarVO7
approved these changes
May 19, 2026
anurag-atlan
added a commit
that referenced
this pull request
May 19, 2026
After DISTR-456 (#1774) landed, build-and-publish on consumer apps that haven't opted into the gate started skipping 'Publish to Marketplace' even though all upstream jobs (prepare, build, merge, security-scan) succeeded. Root cause: GitHub Actions implicit skip-propagation. When unit-tests-gate is skipped (input not configured), the skipped status cascades through the needs chain to any downstream job whose if: lacks an explicit status function. publish was the lone job with 'if: inputs.publish' — every other downstream job already uses '!failure() && !cancelled() && needs.X.result == 'success'' which bypasses the implicit cascade. Fix: bring publish's if: into the same shape as app-deployment-dispatcher and the other gated jobs. Refs DISTR-456
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lands the unit-tests gate for
build-and-publish-app.yaml— first slice of DISTR-456 — App Certification Framework.Refuses to publish a version unless the app's unit-tests workflow has run successfully on the publish SHA. The gate runs before any build minutes are spent so a failing test surfaces at minute 0 rather than after the multi-arch build.
Complements the in-flight cert framework on the marketplace side (atlanhq/global-marketplace#198) — image-side checks (image_vuln / secret_leak / sast_scan) run server-side via Temporal; this PR adds the CI-side gate that prevents broken-tests builds from ever reaching that pipeline.
2-phase rollout — this PR is phase 1 (safe to land today)
unit_tests_workflow_fileskip the gate. Apps that have set it run the gate, get a warning if tests fail, but publish still proceeds.unit_tests_workflow_fileto required AND restores hard-block on gate failure. Laggard apps that haven't onboarded by then are publish-blocked until they do.Combined effect of phase 1: no app's publish workflow will fail because of this gate, even after merging consumer PRs for apps with currently-broken tests. They'll see warnings on the workflow run — useful signal — but publish proceeds.
72 consumer draft PRs are already raised on the
anuragbadoni/distr-456-unit-tests-gatebranch. See the consumer-PR search.How it works
The gate tracks runs by ID, not by check name — apps don't need any particular naming convention on their test jobs.
Caller contract (when opting in)
To opt an app into the gate, set the input in
build-and-publish.yaml:And declare
workflow_dispatch:in the tests workflow'son:block so the gate can fire fresh runs on commits that haven't been independently tested.See
docs/standards/unit-tests-gate.mdfor the full contract + reference template.Behaviour matrix (phase 1)
workflow_dispatch:Validation
Tested end-to-end on two real apps via test branches (not merged):
checks.yml)PR Checks(4 jobs), all passed, publish proceededRollout (recommended sequence)
anuragbadoni/distr-456-unit-tests-gatebranch at each consumer's own pace. Each PR setsunit_tests_workflow_file:and addsworkflow_dispatch:to the tests workflow. Apps with currently-failing tests can be safely merged — they'll emit warnings but won't break.unit_tests_workflow_fileto required + restores hard-block on gate failure.What's in the diff
.github/workflows/build-and-publish-app.yamlunit_tests_workflow_file(default"").checks: readpermission for the gate's API calls.unit-tests-gatejob (resolve → find-or-dispatch → wait → warn), gated byif: inputs.publish && inputs.unit_tests_workflow_file != ''.::warning::, does notexit 1. PR refactor(ci): enforce unit-tests gate — workflow_file required (DISTR-456 follow-up) #1788 reverts this.prepare.needsextended to[validate-channel, unit-tests-gate]. The existingif: !failure() && !cancelled()already handles the skipped-on-build-only and skipped-on-opt-out cases correctly.docs/standards/unit-tests-gate.md(new).github/workflows/unit-tests.yml.No third-party actions added. The gate uses inline
gh api/gh run view/gh workflow runcalls.Out of scope (follow-ups)
test_coverageactivity (Aditya's scope)