|
| 1 | +<!DOCTYPE html> |
| 2 | +<html> |
| 3 | + <head> |
| 4 | + <meta charset="utf-8"> |
| 5 | + <meta name="generator" content="pandoc"> |
| 6 | + <title>Software Carpentry: Testing</title> |
| 7 | + <link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" /> |
| 8 | + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> |
| 9 | + <link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap.css" /> |
| 10 | + <link rel="stylesheet" type="text/css" href="css/bootstrap/bootstrap-theme.css" /> |
| 11 | + <link rel="stylesheet" type="text/css" href="css/swc.css" /> |
| 12 | + <link rel="alternate" type="application/rss+xml" title="Software Carpentry Blog" href="http://software-carpentry.org/feed.xml"/> |
| 13 | + <meta charset="UTF-8" /> |
| 14 | + <!-- HTML5 shim, for IE6-8 support of HTML5 elements --> |
| 15 | + <!--[if lt IE 9]> |
| 16 | + <script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script> |
| 17 | + <![endif]--> |
| 18 | + </head> |
| 19 | + <body class="lesson"> |
| 20 | + <div class="container card"> |
| 21 | + <div class="banner"> |
| 22 | + <a href="http://software-carpentry.org" title="Software Carpentry"> |
| 23 | + <img alt="Software Carpentry banner" src="img/software-carpentry-banner.png" /> |
| 24 | + </a> |
| 25 | + </div> |
| 26 | + <article> |
| 27 | + <div class="row"> |
| 28 | + <div class="col-md-10 col-md-offset-1"> |
| 29 | + <a href="index.html"><h1 class="title">Testing</h1></a> |
| 30 | + <h2 class="subtitle">Running Tests with pytest</h2> |
| 31 | + <section class="objectives panel panel-warning"> |
| 32 | +<div class="panel-heading"> |
| 33 | +<h2 id="learning-objectives"><span class="glyphicon glyphicon-certificate"></span>Learning Objectives</h2> |
| 34 | +</div> |
| 35 | +<div class="panel-body"> |
| 36 | +<ul> |
| 37 | +<li>Understand how to run a test suite using the pytest framework</li> |
| 38 | +<li>Understand how to read the output of a pytest test suite</li> |
| 39 | +</ul> |
| 40 | +</div> |
| 41 | +</section> |
| 42 | +<p>We created a suite of tests for our mean function, but it was annoying to run them one at a time. It would be a lot better if there were some way to run them all at once, just reporting which tests fail and which succeed.</p> |
| 43 | +<p>Thankfully, that exists. Recall our tests:</p> |
| 44 | +<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="im">from</span> mean <span class="im">import</span> <span class="op">*</span> |
| 45 | + |
| 46 | +<span class="kw">def</span> test_ints(): |
| 47 | + num_list <span class="op">=</span> [<span class="dv">1</span>,<span class="dv">2</span>,<span class="dv">3</span>,<span class="dv">4</span>,<span class="dv">5</span>] |
| 48 | + obs <span class="op">=</span> mean(num_list) |
| 49 | + exp <span class="op">=</span> <span class="dv">3</span> |
| 50 | + <span class="cf">assert</span> obs <span class="op">==</span> exp |
| 51 | + |
| 52 | +<span class="kw">def</span> test_zero(): |
| 53 | + num_list<span class="op">=</span>[<span class="dv">0</span>,<span class="dv">2</span>,<span class="dv">4</span>,<span class="dv">6</span>] |
| 54 | + obs <span class="op">=</span> mean(num_list) |
| 55 | + exp <span class="op">=</span> <span class="dv">3</span> |
| 56 | + <span class="cf">assert</span> obs <span class="op">==</span> exp |
| 57 | + |
| 58 | +<span class="kw">def</span> test_double(): |
| 59 | + <span class="co"># This one will fail in Python 2</span> |
| 60 | + num_list<span class="op">=</span>[<span class="dv">1</span>,<span class="dv">2</span>,<span class="dv">3</span>,<span class="dv">4</span>] |
| 61 | + obs <span class="op">=</span> mean(num_list) |
| 62 | + exp <span class="op">=</span> <span class="fl">2.5</span> |
| 63 | + <span class="cf">assert</span> obs <span class="op">==</span> exp |
| 64 | + |
| 65 | +<span class="kw">def</span> test_long(): |
| 66 | + big <span class="op">=</span> <span class="dv">100000000</span> |
| 67 | + obs <span class="op">=</span> mean(<span class="bu">range</span>(<span class="dv">1</span>,big)) |
| 68 | + exp <span class="op">=</span> big<span class="op">/</span><span class="fl">2.0</span> |
| 69 | + <span class="cf">assert</span> obs <span class="op">==</span> exp |
| 70 | + |
| 71 | +<span class="kw">def</span> test_complex(): |
| 72 | + <span class="co"># given that complex numbers are an unordered field</span> |
| 73 | + <span class="co"># the arithmetic mean of complex numbers is meaningless</span> |
| 74 | + num_list <span class="op">=</span> [<span class="dv">2</span> <span class="op">+</span> 3j, <span class="dv">3</span> <span class="op">+</span> 4j, <span class="op">-</span><span class="dv">32</span> <span class="op">-</span> 2j] |
| 75 | + obs <span class="op">=</span> mean(num_list) |
| 76 | + exp <span class="op">=</span> <span class="va">NotImplemented</span> |
| 77 | + <span class="cf">assert</span> obs <span class="op">==</span> exp</code></pre></div> |
| 78 | +<p>Once these tests are written in a file called <code>test_mean.py</code>, the command <code>py.test</code> can be called from the directory containing the tests:</p> |
| 79 | +<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash">$ <span class="kw">py.test</span></code></pre></div> |
| 80 | +<pre class="output"><code>collected 5 items |
| 81 | + |
| 82 | +test_mean.py ....F |
| 83 | + |
| 84 | +================================== FAILURES =================================== |
| 85 | +________________________________ test_complex _________________________________ |
| 86 | + |
| 87 | + def test_complex(): |
| 88 | + # given that complex numbers are an unordered field |
| 89 | + # the arithmetic mean of complex numbers is meaningless |
| 90 | + num_list = [2 + 3j, 3 + 4j, -32 - 2j] |
| 91 | + obs = mean(num_list) |
| 92 | + exp = NotImplemented |
| 93 | +> assert obs == exp |
| 94 | +E assert (-9+1.6666666666666667j) == NotImplemented |
| 95 | + |
| 96 | +test_mean.py:34: AssertionError |
| 97 | +===================== 1 failed, 4 passed in 2.71 seconds ======================</code></pre> |
| 98 | +<p>In the above case, the pytest package ‘sniffed-out’ the tests in the directory and ran them together to produce a report of the sum of the files and functions matching the regular expression <code>[Tt]est[-_]*</code>.</p> |
| 99 | +<p>The major boon a testing framework provides is exactly that, a utility to find and run the tests automatically. With pytest, this is the command-line tool called <code>py.test</code>. When <code>py.test</code> is run, it will search all directories below where it was called, find all of the Python files in these directories whose names start or end with <code>test</code>, import them, and run all of the functions and classes whose names start with <code>test</code> or <code>Test</code>. This automatic registration of test code saves tons of human time and allows us to focus on what is important: writing more tests.</p> |
| 100 | +<p>When you run <code>py.test</code>, it will print a dot (<code>.</code>) on the screen for every test that passes, an <code>F</code> for every test that fails or where there was an unexpected error. In rarer situations you may also see an <code>s</code> indicating a skipped tests (because the test is not applicable on your system) or a <code>x</code> for a known failure (because the developers could not fix it promptly). After the dots, pytest will print summary information.</p> |
| 101 | +<section class="challenge panel panel-success"> |
| 102 | +<div class="panel-heading"> |
| 103 | +<h2 id="fix-the-failing-code"><span class="glyphicon glyphicon-pencil"></span>Fix The Failing Code</h2> |
| 104 | +</div> |
| 105 | +<div class="panel-body"> |
| 106 | +<p>Without changing the tests, alter the mean.py file from the previous section until it passes. When it passes, <code>py.test</code> will produce results like the following:</p> |
| 107 | +<div class="sourceCode"><pre class="sourceCode bash"><code class="sourceCode bash">$ <span class="kw">py.test</span></code></pre></div> |
| 108 | +<p><sub>~</sub> {.output} collected 5 items</p> |
| 109 | +<p>test_mean.py …..</p> |
| 110 | +<p>========================== 5 passed in 2.68 seconds ===========================</p> |
| 111 | +</div> |
| 112 | +</section> |
| 113 | +<p>As we write more code, we would write more tests, and pytest would produce more dots. Each passing test is a small, satisfying reward for having written quality scientific software. Now that you know how to write tests, let’s go into what can go wrong.</p> |
| 114 | + </div> |
| 115 | + </div> |
| 116 | + </article> |
| 117 | + <div class="footer"> |
| 118 | + <a class="label swc-blue-bg" href="http://software-carpentry.org">Software Carpentry</a> |
| 119 | + <a class="label swc-blue-bg" href="https://github.com/swcarpentry/lesson-template">Source</a> |
| 120 | + <a class=" label swc-blue-bg" href=" mailto:[email protected]" >Contact </a> |
| 121 | + <a class="label swc-blue-bg" href="LICENSE.html">License</a> |
| 122 | + </div> |
| 123 | + </div> |
| 124 | + <!-- Javascript placed at the end of the document so the pages load faster --> |
| 125 | + <script src="http://software-carpentry.org/v5/js/jquery-1.9.1.min.js"></script> |
| 126 | + <script src="css/bootstrap/bootstrap-js/bootstrap.js"></script> |
| 127 | + </body> |
| 128 | +</html> |
0 commit comments