bin/verify-exercises-in-docker

[keiravillekode]

This script is useful for testing exercises against a test runner.

All 23 exercises pass. 🎉

[codingthat]

Thanks @keiravillekode. At first glance I'm not sure about the jq inclusion here given that we intentionally removed it in #102 and #104, but I will have a closer look when I can.

[keiravillekode]

We don't need jq here for gdscript, now omitted.

It is useful on tracks when an occasional exercise has more than one file:

https://github.com/exercism/rust/blob/main/exercises/practice/book-store/.meta/config.json

https://github.com/exercism/java/blob/main/exercises/practice/error-handling/.meta/config.json

[codingthat]

Hi again @keiravillekode , sorry for the delay, and thanks for your update.

I've been away from this project for awhile pending some blockers. On closer inspection, following the readme from this repo to the gdscript-test-runner repo, 2/3 of the proposed functionality here seems to already be implemented over there, albeit in two separate scripts.

https://github.com/exercism/gdscript-test-runner#run-the-test-runner-on-an-exercise-using-docker
https://github.com/exercism/gdscript-test-runner#run-the-tests-using-docker

The part missing is

# Example: verify all exercises against specified test runner
# bin/verify-exercises-in-docker -i my-local-image

I'm hesitant to duplicate the functionality between the two repos, but maybe the recommended approach has changed since last July when we were trying to get this track off the ground? I welcome your thoughts / clarification of intent either way. Thanks again.

[keiravillekode]

Suppose a gdscript maintainer would like to test the example raindrops solution against the test runner.

I think

cd .../gdscript
bin/verify-exercises-in-docker raindrops

is equivalent to something like

cd .../gdscript
mkdir -p /tmp/gdscript-001/output
cp -r exercises/practice/raindrops/ /tmp/gdscript-001
cp exercises/practice/raindrops/.meta/example.gd /tmp/gdscript-001/raindrops/raindrops.gd

cd .../gdscript-test-runner
./bin/run-in-docker.sh raindrops /tmp/gdscript-001/raindrops /tmp/gdscript-001/output
cat /tmp/gdscript-001/output/results.json

On tracks I maintain, I find this very useful: When a new language version is released, I can check all a track's exercises against a proposed test runner update.

Nothing in the PR is intended to replace what you already have.

[codingthat]

Nifty! Thanks! And thank you for your patience in explaining. 🙏

One last tiny tweak — if the image isn't found, it errors in a loop. Just before verify_exercises, something like:

docker image inspect "${image}" >/dev/null 2>&1 ||
    die "Docker image '${image}' not found"

...makes for nicer UX in that case.

[codingthat]

(If you prefer, we can merge this as-is and I can tweak it afterwards, maybe adding a reference to the new script to the readme too.)

[keiravillekode]

I'll leave the README edit to you