Select the tests that are in all of several groups

[sebastianbergmann]

Groups can be combined with --group and --exclude-group today, but only one way: a test is selected when it is in any of the groups that were named. There is no way to ask for the tests that are in all of them.

This proposes to add support for separating group names with + to select the tests that are in every one of them:

$ phpunit --group database+slow

This runs only the tests that are in the group database and in the group slow. A test that is in only one of the two is not run.

What you can do with it

Any number of groups can be combined:

$ phpunit --group database+slow+flaky

A combination can be used wherever a single group name can be used, so both ways of combining groups can be mixed. Each value of the option is a set of groups a test has to be in, and a test is selected when it matches at least one of those values:

$ phpunit --group database+slow --group smoke

This runs the tests that are in both database and slow, as well as the tests that are in smoke.

--exclude-group works the same way:

$ phpunit --exclude-group database+slow

This runs everything except the tests that are in both database and slow. A test that is only in database, or only in slow, still runs.

The <groups> element of the XML configuration file accepts the same syntax:

<groups>
  <include>
    <group>database+slow</group>
  </include>
</groups>

What this means for group names

+ cannot be both an operator and an ordinary character in a group name. A group named database+slow can no longer be selected by that name: --group database+slow now asks for the tests that are in database and in slow.

So that this cannot happen silently, PHPUnit emits a warning where such a group name is declared: with the Group attribute, or with the groups attribute of a <directory> or <file> element in the XML configuration file. The group is still assigned to the test; only selecting it by name is no longer possible.

A + is only treated as an operator between two group names. +name, name+, and + remain ordinary group names and can be selected as before, so a name is only affected when it has a + between two other characters.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.46%. Comparing base (c5c3c74) to head (f415f97).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #6958   +/-   ##
=========================================
  Coverage     99.46%   99.46%           
- Complexity     9614     9639   +25     
=========================================
  Files           936      937    +1     
  Lines         29306    29374   +68     
=========================================
+ Hits          29150    29218   +68     
  Misses          156      156           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.