TestingCombinations.md

Testing Combinations

Contents

• The Basics
  • Passing in a Reporter
• Code samples

The Basics

You can use CombinationApprovals::verifyAllCombinations to test the content of multiple containers.

This makes a kind of approval test matrix, automatically testing all combinations of a set of inputs. It's a powerful way to quickly get very good test coverage.

In this small example, all combinations of {"hello", "world"} and {1, 2, 3} are being used:

TEST_CASE("YouCanVerifyCombinationsOf2") {
    std::vector<std::string> v{"hello", "world"};
    std::vector<int> numbers{1, 2, 3};
    CombinationApprovals::verifyAllCombinations(
            [](std::string s, int i){return std::make_pair(s, i);},
            v,
            numbers);
}

^{snippet source / anchor}

The format is carefully chosen to show both inputs and outputs, to make the test results easy to interpret. The output looks like this:

(hello, 1) => (hello, 1)
(hello, 2) => (hello, 2)
(hello, 3) => (hello, 3)
(world, 1) => (world, 1)
(world, 2) => (world, 2)
(world, 3) => (world, 3)

^{snippet source / anchor}

For advice on effective formatting, see To String. As you write out larger volumes of data in your approval files, experience has shown that the choice of layout of text in approval files can make a big difference to maintainability of tests, when failures occur.

Passing in a Reporter

Note: Because of the limitations of C++, the Reporter is not optional at the end, as in the rest of Approval Tests. Instead, you can optionally add a Reporter as the second argument.

Code samples

CombinationApprovals::verifyAllCombinations(
        []( const std::string& input1, const int input2, const double input3)
        {
            return functionThatReturnsSomethingOutputStreamable(input1, input2, input3);
        }, // This is the converter function
        listOfInput1s,
        listOfInput2s,
        listOfInput3s);

^{snippet source / anchor}

If you are using C++14 or above, you can simplify this by using auto or auto& for the lambda parameters:

CombinationApprovals::verifyAllCombinations(
        []( auto& input1, auto& input2, auto& input3)
        {
            return functionThatReturnsSomethingOutputStreamable(input1, input2, input3);
        }, // This is the converter function
        listOfInput1s,
        listOfInput2s,
        listOfInput3s);

^{snippet source / anchor}

---

Back to User Guide