Run fixtures when working with non-python tests

[sashko1988]

How can someone run fixtures when working with non-python tests?

https://docs.pytest.org/en/stable/example/nonpython.html#non-python-tests Let's take that example and add a simple session scope fixture with autouse=True. How to make pytest execute that fixture?

conftest.py example:

from __future__ import annotations

import pytest


def pytest_collect_file(parent, file_path):
    if file_path.suffix == ".yaml" and file_path.name.startswith("test"):
        return YamlFile.from_parent(parent, path=file_path)


class YamlFile(pytest.File):
    def collect(self):
        # We need a yaml parser, e.g. PyYAML.
        import yaml

        raw = yaml.safe_load(self.path.open(encoding="utf-8"))
        for name, spec in sorted(raw.items()):
            yield YamlItem.from_parent(self, name=name, spec=spec)


class YamlItem(pytest.Item):
    def __init__(self, *, spec, **kwargs):
        super().__init__(**kwargs)
        self.spec = spec

    def runtest(self):
        for name, value in sorted(self.spec.items()):
            # Some custom test execution (dumb example follows).
            if name != value:
                raise YamlException(self, name, value)

    def repr_failure(self, excinfo):
        """Called when self.runtest() raises an exception."""
        if isinstance(excinfo.value, YamlException):
            return "\n".join(
                [
                    "usecase execution failed",
                    "   spec failed: {1!r}: {2!r}".format(*excinfo.value.args),
                    "   no further details known at this point.",
                ]
            )
        return super().repr_failure(excinfo)

    def reportinfo(self):
        return self.path, 0, f"usecase: {self.name}"


class YamlException(Exception):
    """Custom exception for error reporting."""


@pytest.fixture(autouse=True, scope="session")
def print_name(request):
    print("-*-* executing session -*-*")

Even if I add fixtures manually in the item __init__, they aren't called anyway:

class YamlItem(pytest.Item):
    def __init__(self, *, spec, **kwargs):
        super().__init__(**kwargs)
        self.spec = spec
        self.fixtureinfo = self.session._fixturemanager.getfixtureinfo(self, None, None)
        self.fm =self.session._fixturemanager
        self.fixturenames = self.fixtureinfo.names_closure

[RonnyPfannschmidt]

Currently fixtures are only implemented in python nodes

[nicoddemus]

I wonder if we could move it to item in general

Yeah I wish for that too, but probably needs to be done cautiously, as we plan to eventually provide hooks and what not to customize the fixture mechanism. 😬

[sashko1988]

Do you need any feature requests/issues for this work?

[nicoddemus]

We already have issues to track this, thanks (what we lack is find time to work on them, apparently hehehe).

[sashko1988]

@nicoddemus can you share them, please?

[sashko1988]

@nicoddemus, @RonnyPfannschmidt, any updates on the public API for setting fixtures for non-Python tests? If you have a GitHub issue, I can look into it.

[dongfangtianyu]

What a coincidence, I’ve done something similar before!

In short, you need to:

1. Obtain the fixturemanager
2. Delegate it to analyze and obtain the fixture results

It looks like you’ve only done step 1, but not step 2.

You can either inherit pytest.Function or implement this part of the code in your own class.
https://github.com/pytest-dev/pytest/blob/main/src/_pytest/python.py#L1580-L1585

Then, access self._request:

def runtest(self) -> None:
    self._request._fillfixtures()  #  call fixture
    print(self.funcargs)  # has fixture result

[sashko1988]

I missed that answer. As I said, I wanted to avoid a private API because it can change at any minute. But yeah, using that's the only way right now (which we use)

[RonnyPfannschmidt]

I recommended adding a ci pipeline that runs against pytest main

Changes to the area are not often but we do have a backlog item of enabling all items for fixtures