A place to cache linked articles (think custom and personal wayback machine)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

index.md 10KB

4 lat temu
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123
  1. title: Your tests as your specs
  2. url: http://blog.mathieu-leplatre.info/your-tests-as-your-specs.html
  3. hash_url: 9b47ffd334e0f37ebe54377f41dfa4e4
  4. <p>At last, with all this surrounding pressure, you finally decided to write tests.
  5. Now, you always setup TravisCI to run when code change is submitted. You now feel
  6. confident when pull-requests come in, and soon your test suite code coverage will
  7. reach 100%.</p>
  8. <p>Let me be difficult to please here...</p>
  9. <img alt="" class="align-center" src="/images/tests-specs-nitpick.jpg"/>
  10. <p>You can reach clean code heaven with one final little step: <strong>transform the tests
  11. suites into specifications</strong>!</p>
  12. <div class="section" id="bad-pattern">
  13. <h2>Bad pattern</h2>
  14. <p>In many Open Source projects, a great attention is paid to the quality of the
  15. code itself. It is quite often that some code is merged without the tests being
  16. exhaustively reviewed.</p>
  17. <p>In other words, it is not rare that the quality of the tests is not reflecting
  18. the quality of the application code.</p>
  19. <p>The most frequent pattern is that all tests are implemented in the same test case :</p>
  20. <div class="highlight"><pre><span class="k">class</span> <span class="nc">GameTest</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
  21. <span class="k">def</span> <span class="nf">test_update</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  22. <span class="n">game</span> <span class="o">=</span> <span class="n">Game</span><span class="p">()</span>
  23. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">game</span><span class="o">.</span><span class="n">played</span><span class="p">,</span> <span class="bp">False</span><span class="p">)</span>
  24. <span class="n">game</span><span class="o">.</span><span class="n">score</span> <span class="o">=</span> <span class="mi">4</span>
  25. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">game</span><span class="o">.</span><span class="n">played</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span>
  26. <span class="n">game</span><span class="o">.</span><span class="n">player</span> <span class="o">=</span> <span class="s">'Jean-Louis'</span>
  27. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="nb">unicode</span><span class="p">(</span><span class="n">game</span><span class="p">),</span> <span class="s">'Jean-Louis (score: 4)'</span><span class="p">)</span>
  28. <span class="c"># See bug #2780</span>
  29. <span class="n">game</span><span class="o">.</span><span class="n">score</span> <span class="o">=</span> <span class="bp">None</span>
  30. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="nb">unicode</span><span class="p">(</span><span class="n">game</span><span class="p">),</span> <span class="s">'Jean-Louis'</span><span class="p">)</span>
  31. </pre></div>
  32. <p>Writing tests like this has many drawbacks:</p>
  33. <ul class="simple">
  34. <li>One has to read the entire test to understand the expected behaviour of the code;</li>
  35. <li>If the test fails, it will be hard to find out what part of the code has failed precisely;</li>
  36. <li>Refactoring the code will probably mean to rewrite the entire test since instructions are
  37. inter-dependant;</li>
  38. <li>The <tt class="docutils literal">TestCase</tt> and <tt class="docutils literal">setUp()</tt> notions are underused.</li>
  39. </ul>
  40. </div>
  41. <div class="section" id="better-pattern">
  42. <h2>Better pattern</h2>
  43. <p>A simple way to improve the quality of the tests is to see them as specifications.
  44. After all, it makes sense, you add some code for a reason! The tests are not only
  45. here to prevent regressions, but also to explicit the expected behaviour of the application!</p>
  46. <p>I believe that many projects would take great benefits if following this approach .</p>
  47. <div class="highlight"><pre><span class="c"># test_game.py</span>
  48. <span class="k">class</span> <span class="nc">InitializationTest</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
  49. <span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  50. <span class="bp">self</span><span class="o">.</span><span class="n">game</span> <span class="o">=</span> <span class="n">Game</span><span class="p">()</span>
  51. <span class="k">def</span> <span class="nf">test_default_played_status_is_false</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  52. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="o">.</span><span class="n">played</span><span class="p">,</span> <span class="bp">False</span><span class="p">)</span>
  53. <span class="k">class</span> <span class="nc">UpdateTest</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
  54. <span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  55. <span class="bp">self</span><span class="o">.</span><span class="n">game</span> <span class="o">=</span> <span class="n">Game</span><span class="p">()</span>
  56. <span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="o">.</span><span class="n">player</span> <span class="o">=</span> <span class="s">'Jean-Louis'</span>
  57. <span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="o">.</span><span class="n">score</span> <span class="o">=</span> <span class="mi">4</span>
  58. <span class="k">def</span> <span class="nf">test_played_status_is_true_if_score_is_set</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  59. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="o">.</span><span class="n">played</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span>
  60. <span class="k">def</span> <span class="nf">test_string_representation_is_player_with_score_if_played</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  61. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="nb">unicode</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="p">),</span> <span class="s">'Jean-Louis (score: 4)'</span><span class="p">)</span>
  62. <span class="k">def</span> <span class="nf">test_string_representation_is_only_player_if_not_played</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
  63. <span class="c"># See bug #2780</span>
  64. <span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="o">.</span><span class="n">score</span> <span class="o">=</span> <span class="bp">None</span>
  65. <span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="nb">unicode</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">game</span><span class="p">),</span> <span class="s">'Jean-Louis'</span><span class="p">)</span>
  66. </pre></div>
  67. <p>Writing tests like this has many advantages:</p>
  68. <ul class="simple">
  69. <li>Each case isolates a situation;</li>
  70. <li>Each test is an aspect of the specification;</li>
  71. <li>Each test is independant;</li>
  72. <li>The testing vocabulary is honored: we <em>setup</em> a <em>test case</em>;</li>
  73. <li>If a test fails, it is straightforward to understand what part of the spec
  74. was violated;</li>
  75. <li>Tests that were written when fixing bugs will explicit the expected behaviour
  76. for edge cases.</li>
  77. </ul>
  78. </div>
  79. <div class="section" id="reporting">
  80. <h2>Reporting</h2>
  81. <p>Now that we have explicited the specs, we will want to read them properly.</p>
  82. <p>One of things I like in JavaScript is <a class="reference external" href="http://mochajs.org">Mocha</a>. Appart
  83. from the nice API and the very rich feature set, its default test reporter is
  84. great, tt is colourful and structurally invites you to write tests as specs.</p>
  85. <img alt="" class="align-center" src="/images/tests-specs-mocha.png"/>
  86. <p>In our project, we were using <a class="reference external" href="http://nose.readthedocs.org">nose</a>, so I
  87. decided to write a reporter that would produce the same output as Mocha.</p>
  88. <p>You can install and use it this way:</p>
  89. <pre class="literal-block">
  90. $ pip install nose-mocha-reporter
  91. $ nosetests --with-mocha-reporter yourpackage/
  92. </pre>
  93. <p>It will produce the following output:</p>
  94. <img alt="" class="align-center" src="/images/tests-specs-nose-reporter.png"/>
  95. <p>It takes the tests suites and extract the names as readable strings:</p>
  96. <ul class="simple">
  97. <li><tt class="docutils literal">tests/core/test_game.py</tt> → <tt class="docutils literal">CORE GAME</tt></li>
  98. <li><tt class="docutils literal">class InitializationTest(TestCase)</tt> → <tt class="docutils literal">Initialization</tt></li>
  99. <li><tt class="docutils literal">def test_played_status_is_true_if_score_is_set</tt> → <tt class="docutils literal">Played status is true if score is set</tt></li>
  100. </ul>
  101. <p>It also mesures the execution time of each test in order highlight the slow ones.</p>
  102. <p>To conclude, this reporter has a pretty modest objective: remind you that <strong>the tests
  103. you write should be read as specifications</strong> !</p>
  104. </div>
  105. <div class="section" id="special-thanks">
  106. <h2>Special thanks!</h2>
  107. <p>I'm very grateful to <a class="reference external" href="http://antoine.cezar.fr/">Antoine</a> and
  108. <a class="reference external" href="http://alexmarandon.com/">Alex</a> that showed me the light on
  109. this. Since they might not be conscious of the influence they had on me,
  110. I jump on the occasion to thank them loudly :)</p>