How to fail rather than emit warning

[jfoust-zymergen]

I've noticed that a misconfigured annotation will emit a warning like the following:

WARNING: cannot execute 'test_foo' relative to others: 'other/test.py::thats_required' - ignoring the marker.

Is there a way to configure pytest-order such that it fails rather than emits a warning when it detects an error like this?

[mrbean-bremen]

No, there isn't, though that could be added - probably via some kind of strict option, that would cause an exception instead of a warning in these cases. I'm open to suggestions...

[jfoust-zymergen]

@mrbean-bremen thanks for the quick response -- is there a way to get pytest-order to skip tests that are ordered after skipped/failed tests? I understand this isn't explicitly the focus on pytest-order but pytest-dependency is currently is an unstable/unreliable state.

[mrbean-bremen]

Hm, I explicitely added the option --order-dependencies to sort the tests with dependency markers, so that the two plugins could work together. I wasn't aware of that problem in pytest-dependency, I will probably have a look at that first.
I would rather like to get pytest-dependencies fixed before integrating its functionality into pytest-order (which could be difficult anyway).

[mrbean-bremen]

So far, I could not reproduce the problem. I suspect that there is some setup involved that is not obvious... I may have another look over the weekend if I find the time.

[jfoust-zymergen]

@mrbean-bremen in our case the issue is manifesting when attempting to depend on tests in other file/package. I was applying scope=session to the dependency markers.

[mrbean-bremen]

@jfoust-zymergen - I had a look at that issue, and it is not clear if it is a bug at all, and if yes, how to reproduce it. The original issue certainly isn't (I wrote a comment there). I would suggest you write a separate issue there if you have a reproducible case that does not work. Also, is that related to your original question, or just a separate one?
As for the original question: I may add a respective feature (exception instead of warning), maybe you can add a bit of the rationale there, and how it should look? You also may write that as a separate feature request.