-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added Xfail and Mayfail attributes to city runner #686
Merged
coldav
merged 7 commits into
uxlfoundation:main
from
coldav:colin/add_xfail_city_runner
Feb 25, 2025
Merged
Added Xfail and Mayfail attributes to city runner #686
coldav
merged 7 commits into
uxlfoundation:main
from
coldav:colin/add_xfail_city_runner
Feb 25, 2025
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8142c1d
to
be7fd1f
Compare
hvdijk
reviewed
Feb 21, 2025
hvdijk
reviewed
Feb 21, 2025
hvdijk
reviewed
Feb 21, 2025
hvdijk
reviewed
Feb 21, 2025
hvdijk
reviewed
Feb 21, 2025
hvdijk
reviewed
Feb 21, 2025
7029900
to
3b259da
Compare
hvdijk
reviewed
Feb 24, 2025
hvdijk
reviewed
Feb 24, 2025
hvdijk
reviewed
Feb 24, 2025
hvdijk
reviewed
Feb 24, 2025
3b259da
to
c78de58
Compare
hvdijk
reviewed
Feb 25, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks pretty good! Some odd whitespace and perhaps slightly more verbose naming and complicated logic than needed, left some suggestions for that, but the logic for the results looks right as far as I can see
Support attribute Xfail and Mayfail for city runner. These can be set as additional elements in the csv file. Xfail means we expect it to fail and will give a failing return code if it unexpectedly pass, otherwise it does not contribute to an overall fail. Mayfail means it sometimes fails and will not contribute to a failing return code. Unlike some of the other attributes, these cannot be set with additional csv files, as it was felt this leads to overcomplexity. Instead it is expected that some form of csv combination script could be used alongside this.
27ce583
to
9e3a667
Compare
hvdijk
approved these changes
Feb 25, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Support attribute Xfail and Mayfail for city runner and an add a --override option to override entries in the main csv file.
Reason for change
We want to be able to run tests that we know fail, but we don't want an overall fail.
Description of change
Added xfail and Mayfail as attributes to city runner.
Xfail means we expect it to fail and will give a failing return code if it unexpectedly pass, otherwise it does not contribute to an overall fail.
Mayfail means it sometimes fails and will not contribute to a failing return code.
Unlike some of the other attributes, these cannot be set with additional csv files, as it was felt this leads to overcomplexity. Instead it is expected that some form of csv combination script could be used alongside this.
Added --override option. This provides an additional csv file which can override entries in the main csv file if it matches the first two values in an entry. This is useful for overriding attributes such as Xfail.
Anything else we should know?
The following is example output
dump.txt
Checklist