Assertion rewrites not applied when running pytest within an embedded Python environment

[bradleyharden]

I'm trying to use pytest in an application that uses Python as an embedded scripting language. Essentially, I have the application run a script which calls pytest.main(). At first, I thought everything was going well, except for the issue mentioned in #12453. All my tests were passing. But now I realize that my tests will pass in virtually every scenario. Even the following test passes:

def test_failing():
    assert False

Yet if I change assert False to raise AssertionError, the test will fail. It's as if the assertions are not be rewritten at all. Does anyone know why that might be the case?

To address #12453, I've been trying to figure out how to use importlib.reload, but I still see this behavior even if I restart the application and delete .pytest_cache. I don't think that has anything to do with it.

[The-Compiler]

I'm confused why you think this is related to assertion rewriting. An assert False should cause an AssertionError in Python - rewritten or not, and even outside of pytest. All rewriting does is showing additional context when the assertion fails.

It sounds like something is broken in your Python environment. Possibly you're using PYTHONOPTIMIZE / -O?

[bradleyharden]

@The-Compiler, now that you point it out, my chain of reasoning isn't very solid. I knew pytest rewrote assertions, and my assertion seemed to not be working as expected, so I assumed it was somehow to blame. But what I didn't do was check that assert False ever raises an AssertionError in my Python environment. And indeed it does not.

I vaguely recall hearing about an option that disabled asserts in Python, but I don't think I would have come to that without your suggestion. I'll dig in and see what's going on. Thanks for the suggestion.

[bradleyharden]

I can only assume that the interpreter is invoked with -O, as __debug__ is False. However, I don't immediately see any way to undue that for pytest. I'm trying to figure out an alternative, but I thought I might ask here as well, in case someone has an idea or advice.

[bradleyharden]

I think this deserves a new question with an accurate title.

[bradleyharden]

See #12460

[bradleyharden]

@The-Compiler, my initial reasoning was wrong, but I'm back to questioning whether rewriting is applied or not. It seems like pytest should have saved me here, without me even realizing it.

Doesn't rewriting transform assert statements into raise AssertionError equivalents? The former returns None in my system, while the latter raises an exception. If rewriting were being applied, wouldn't assert False be converted to something like this?

@py_assert1 = False
if (not @py_assert1):
    raise AssertionError(...)
del @py_assert1

I would expect this code to fail the test in my system, yet it doesn't.