claude: Make Windows smoke tests fail-fast like Linux

[gordonwoodhull]

This is a small quality-of-life change to the Windows tests, that I've been using in my branch for a while.

I find it convenient to see the error immediately when checking results in CI, rather than it printing a bunch of successes afterward. (And possibly more failures, I acknowledge.)

I think I got distracted one evening and asked Claude if it could figure out why this is, and it diagnosed it as "failing fast" in Linux but not on Windows. So I tried its suggestion.

@cderv, your choice whether to take or leave it. Maybe there are other reasons I don't know for logging errors and then continuing?

Description

When a test fails on Windows, exit immediately instead of continuing through remaining tests in the bucket. This matches Linux behavior (bash -e flag) and keeps test failures at the end of the log where they're easy to find, instead of buried under 1000+ lines of subsequent test output.

[posit-snyk-bot]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scanner	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[cderv]

Thanks.

Just to be clear, are linux tests failing faster ?

I don't see exactly the same as claude saying:

"failing fast" in Linux but not on Windows.

Code is

quarto-cli/.github/workflows/test-smokes.yml

Lines 260 to 269 in af0bb42

	run: |
	haserror=0
	readarray -t my_array < <(echo '${{ inputs.buckets }}' | jq -rc '.[]')
	for file in "${my_array[@]}"; do
	echo ">>> ./run-tests.sh ${file}"
	shopt -s globstar && ./run-tests.sh $file
	status=$?
	[ $status -eq 0 ] && echo ">>> No error in this test file" || haserror=1
	done
	[ $haserror -eq 0 ] && echo ">>> All tests passed" || exit 1

We do

[ $status -eq 0 ] && echo ">>> No error in this test file" || haserror=1

so if failing tests, then we set haserror to 1 and we continue the loop

Then when done, we exit if there was one error

[ $status -eq 0 ] && echo ">>> No error in this test file" || haserror=1

Do I understand things wrongly?

To me this PR makes Linux and Windows behave differently.

We did Linux and Windows the same, by not failing fast by design for now. This is because that way we get ALL the tests that are failing on a buckets. And we can investigate locally all of them.
Failing fast would mean, we get a failure in the bucket. We solve it locally, we trigger new CI by a commit and then we discover the next failure. It seemed to me not as convenient as knowing all the issues at once.

Now with experience, maybe we don't have that much difference in failure each time, and in PR runs we could indeed fail fast, and then keep non fast fail when CI is running on main.

Anyhow, we can change and fail fast if this is prefered. But we need to adapt Linux test too I believe.

Let's discuss this.