Skip to content

✨ Add --skip-errors option to continue scans on check/probe failures#5117

Open
jimjag wants to merge 1 commit into
ossf:mainfrom
jimjag:feat/skip-errors
Open

✨ Add --skip-errors option to continue scans on check/probe failures#5117
jimjag wants to merge 1 commit into
ossf:mainfrom
jimjag:feat/skip-errors

Conversation

@jimjag

@jimjag jimjag commented Jul 6, 2026

Copy link
Copy Markdown

What kind of change does this PR introduce?

Feature.

What is the current behavior?

If a single check or probe fails with a runtime error (for example a transient GitHub API failure such as a 404 on ListWorkflowRunsByFileName), the entire scan aborts and no results are reported. This is especially disruptive when scanning many repositories with --repos or --org, where one transient failure fails the whole run.

Example:

Check Packaging failed for evergage/mongodb_exporter: internal error: ... 404 Not Found []
Error: one or more checks failed during execution

What is the new behavior (if this is a feature change)?

Adds a --skip-errors option. When set, if one or more checks or probes fail at runtime, Scorecard continues running the remaining checks/probes and reports results for everything that succeeded. Failures are logged to stderr and the command exits with a zero status.

  • Adds the SkipErrors option and --skip-errors CLI flag.

  • Threads skipErrors through scorecard.Run into the probe and raw-data collection paths, so a failure skips only the affected item instead of aborting the run.

  • Suppresses the "one or more checks failed" error (non-zero exit) when --skip-errors is set.

  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

The public scorecard.Run API is unchanged; a new WithSkipErrors option was added and defaults to false, so existing callers (serve, cron, attestor, scdiff) keep their current behavior. The default CLI behavior is also unchanged unless --skip-errors is passed.

Does this PR introduce a user-facing change?

Yes — a new optional --skip-errors CLI flag. Default behavior is unchanged.

Add a `--skip-errors` option that continues a scan when individual checks or probes fail at runtime, reporting results for everything that succeeded instead of aborting the whole run.

Previously, if a single check or probe failed with a runtime error, the
entire scan aborted and no results were reported. This is especially
disruptive when scanning many repositories with --repos or --org, where
one transient API failure fails the whole run.

Add a --skip-errors option so that when one or more checks or probes fail
at runtime, Scorecard continues running the remaining checks/probes and
reports results for everything that succeeded. Failures are logged to
stderr and the command exits with a zero status.

Changes:
- Add SkipErrors option and --skip-errors CLI flag.
- Thread skipErrors through scorecard.Run into the probe and raw-data
  collection paths so a failure skips only the affected item instead of
  aborting the run.
- Do not return the "one or more checks failed" error (non-zero exit)
  when --skip-errors is set.
- Add unit tests for the flag and the skip-vs-abort behavior, and
  document the option in the README.

Signed-off-by: Jim Jagielski <jimjag@gmail.com>
@jimjag jimjag requested a review from a team as a code owner July 6, 2026 20:20
@jimjag jimjag requested review from jeffmendoza and justaugustus and removed request for a team July 6, 2026 20:20
@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant