{"id":218,"date":"2015-07-27T10:33:42","date_gmt":"2015-07-27T10:33:42","guid":{"rendered":"https:\/\/blogs.mathworks.com\/developer\/?p=218"},"modified":"2015-08-04T14:11:32","modified_gmt":"2015-08-04T14:11:32","slug":"addteardown","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/developer\/2015\/07\/27\/addteardown\/","title":{"rendered":"Tear Down This Wall!"},"content":{"rendered":"<div class=\"content\"><p>I once faced a question from a friend who was perusing MATLAB documentation while implementing a test suite  for some production software and found an example test which used the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcase.addteardown.html\"><b><tt>addTeardown<\/tt><\/b><\/a> method of the <b><tt>TestCase<\/tt><\/b> object. This person found themselves wondering why they might want to use it to manage their test fixture, citing that it was unclear where <b><tt>addTeardown<\/tt><\/b> fit with other fixture related features like <b><tt>TestMethodTeardown<\/tt><\/b> and <b><tt>TestClassTeardown<\/tt><\/b>. Alright, what is the deal with <b><tt>addTeardown<\/tt><\/b> and why would we use it? This construct is actually not seen in most xUnit paradigms so it may be unfamiliar to many. However, in fact <b><tt>addTeardown<\/tt><\/b> is more robust and, once you get used to it, actually less difficult than the other teardown routines. However its usage is markedly different from standard xUnit paradigms, so <b><tt>TestClassTeardown<\/tt><\/b> and <b><tt>TestMethodTeardown<\/tt><\/b> were included for standard familiarity.<\/p><p>Before we get too far into this, I am going to mysteriously add a few dummy folders to the path, save the entire path contents, and take a snapshot of these top three folders on the path using the following quick helper function. These will be handy as we explore the topic.<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> showTopThreePathFolders\r\np = strsplit(path, pathsep);\r\ndisp(p(1:3)');\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><pre class=\"codeinput\">addpath(fullfile(pwd, <span class=\"string\">'folderOnOriginalPath1'<\/span>));\r\naddpath(fullfile(pwd, <span class=\"string\">'folderOnOriginalPath2'<\/span>));\r\naddpath(fullfile(pwd, <span class=\"string\">'folderOnOriginalPath3'<\/span>));\r\nblogPostPath = path;\r\nshowTopThreePathFolders;\r\n<\/pre><pre class=\"codeoutput\">    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath1'\r\n\r\n<\/pre><p>Got it? Good, moving on.<\/p><p><b>A House of Order<\/b><\/p><p>The first reason that <b><tt>addTeardown<\/tt><\/b> is more robust is that it gives an order guarantee. The contract is that teardown actions occur in the reverse order to which they were added. When different fixtures change the same state, this prevents the teardown from executing out of order and restoring to the wrong end state. Here is an example demonstrating how this setup\/teardown can run in the wrong order without this feature:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">classdef<\/span> BadFixturesTest &lt; matlab.unittest.TestCase\r\n    <span class=\"keyword\">properties<\/span>\r\n        P1\r\n        P2\r\n    <span class=\"keyword\">end<\/span>\r\n    \r\n    <span class=\"keyword\">methods<\/span>(TestMethodSetup)\r\n        <span class=\"keyword\">function<\/span> addPath1(testCase)\r\n            disp(<span class=\"string\">'Adding path1'<\/span>);\r\n            testCase.P1 = addpath(fullfile(pwd, <span class=\"string\">'path1'<\/span>));\r\n            showTopThreePathFolders;\r\n        <span class=\"keyword\">end<\/span>\r\n        \r\n        <span class=\"keyword\">function<\/span> addPath2(testCase)\r\n            disp(<span class=\"string\">'Adding path2'<\/span>);\r\n            testCase.P2 = addpath(fullfile(pwd, <span class=\"string\">'path2'<\/span>));\r\n            showTopThreePathFolders;\r\n        <span class=\"keyword\">end<\/span>\r\n    <span class=\"keyword\">end<\/span>\r\n    <span class=\"keyword\">methods<\/span>(TestMethodTeardown)\r\n        <span class=\"keyword\">function<\/span> restoreP2(testCase)\r\n            disp(<span class=\"string\">'Restoring path2'<\/span>);\r\n            path(testCase.P2);\r\n            showTopThreePathFolders;\r\n        <span class=\"keyword\">end<\/span>\r\n        \r\n        <span class=\"keyword\">function<\/span> restoreP1(testCase)\r\n            disp(<span class=\"string\">'Restoring path1'<\/span>);\r\n            path(testCase.P1);\r\n            showTopThreePathFolders;\r\n        <span class=\"keyword\">end<\/span> \r\n    <span class=\"keyword\">end<\/span>\r\n    \r\n    <span class=\"keyword\">methods<\/span>(Test)\r\n        <span class=\"keyword\">function<\/span> runTest(~)\r\n        <span class=\"keyword\">end<\/span>\r\n    <span class=\"keyword\">end<\/span>\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>A couple things to note about this and the other examples in this post:<\/p><div><ul><li>This example is contrived and is easy to work around, but this scenario can be encountered in more subtle ways.<\/li><li>The order of when setup\/teardown methods are executed is not guaranteed. Even if they seem consistent now they can change. Therefore, tweaking the order of the methods is not a good solution to this problem.<\/li><li>There are other features in the framework that can be used instead of this code, for example the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.fixtures.pathfixture-class.html\"><b><tt>PathFixture<\/tt><\/b><\/a> and the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.fixtures.suppressedwarningsfixture-class.html\"><b><tt>SuppressedWarningsFixture<\/tt><\/b><\/a>. Interestingly, however, using these fixtures with <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcase.applyfixture.html\"><b><tt>applyFixture<\/tt><\/b><\/a> leverages <b><tt>addTeardown<\/tt><\/b>.<\/li><\/ul><\/div><p>Look at what can happen when the methods are executed. Again, the order is not guaranteed so the order shown is possible in general:<\/p><pre class=\"codeinput\">runtests <span class=\"string\">BadFixturesTest<\/span>;\r\nshowTopThreePathFolders;\r\n<\/pre><pre class=\"codeoutput\">Running BadFixturesTest\r\nAdding path1\r\n    '\/Users\/Shared\/blog\/addTeardown\/path1'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n\r\nAdding path2\r\n    '\/Users\/Shared\/blog\/addTeardown\/path2'\r\n    '\/Users\/Shared\/blog\/addTeardown\/path1'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n\r\nRestoring path1\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath1'\r\n\r\nRestoring path2\r\n    '\/Users\/Shared\/blog\/addTeardown\/path1'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n\r\n.\r\nDone BadFixturesTest\r\n__________\r\n\r\n    '\/Users\/Shared\/blog\/addTeardown\/path1'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n\r\n<\/pre><p>The above test run demonstrates the following:<\/p><div><ol><li>addPath1 --&gt; P1 contains original path, path1 is now on the active path.<\/li><li>addPath2 --&gt; P2 now contains the path with path1 on it, both path1 and path2 are now on the active path.<\/li><li>restoreP1 --&gt; The original path is now the active path.<\/li><li>restoreP2 --&gt; The path stored in addPath2 is now the active path. This path still has path1 on it and is <b>NOT<\/b> the path from the start.<\/li><\/ol><\/div><p>As you can see, the path was clobbered because we had no guarantee of order. One thought may be to use <b><tt>rmpath<\/tt><\/b> instead of <b><tt>path<\/tt><\/b> to restore the state, but the problem with <b><tt>rmpath<\/tt><\/b> is that it might result in removing something from the path that was on the path at the start. Using the <b><tt>path<\/tt><\/b> function allows the test to ignore whether or not the folder was on the path to start. Instead, we can solve this using <b><tt>addTeardown<\/tt><\/b> (and the result is much smaller to boot):<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">classdef<\/span> GoodFixturesTest &lt; matlab.unittest.TestCase\r\n    \r\n    <span class=\"keyword\">methods<\/span>(TestMethodSetup)\r\n        <span class=\"keyword\">function<\/span> addPath1(testCase)\r\n            p = addpath(fullfile(pwd, <span class=\"string\">'path1'<\/span>));\r\n            testCase.addTeardown(@path, p);\r\n        <span class=\"keyword\">end<\/span>\r\n        \r\n        <span class=\"keyword\">function<\/span> addPath2(testCase)\r\n            p = addpath(fullfile(pwd, <span class=\"string\">'path2'<\/span>));\r\n            testCase.addTeardown(@path, p);\r\n        <span class=\"keyword\">end<\/span>\r\n    <span class=\"keyword\">end<\/span>\r\n    \r\n    <span class=\"keyword\">methods<\/span>(Test)\r\n        <span class=\"keyword\">function<\/span> runTest(~)\r\n        <span class=\"keyword\">end<\/span>\r\n    <span class=\"keyword\">end<\/span>\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><pre class=\"codeinput\">path(blogPostPath); <span class=\"comment\">% Restore to the original path<\/span>\r\nruntests <span class=\"string\">GoodFixturesTest<\/span>;\r\nshowTopThreePathFolders;\r\n<\/pre><pre class=\"codeoutput\">Running GoodFixturesTest\r\n.\r\nDone GoodFixturesTest\r\n__________\r\n\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath1'\r\n\r\n<\/pre><p>Now it doesn't matter which setup method is executed first because <b><tt>addTeardown<\/tt><\/b> executes in LIFO order. We have a guarantee that the teardown is executed in the reverse order of setup.<\/p><p><b>Let's Make Exceptions<\/b><\/p><p>OK, now another way to enforce order of setup\/teardown is to write the code as a single setup and teardown function and code the order directly. This works but is not exception safe. We want to guarantee that the state we modify in the test is correctly restored to the original state even in the event of unexpected errors as well as other code flow interruptions like assertion and assumption failures. However this is problematic when there is a lot of fixture setup\/teardown in single methods. Observe the following scenario, this time with a function-based test just to show that <b><tt>addTeardown<\/tt><\/b> can be used there as well:<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> tests = NotExceptionSafeTest\r\ntests = functiontests(localfunctions);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> setup(testCase)\r\ntestCase.TestData.P1 = addpath(fullfile(pwd, <span class=\"string\">'path1'<\/span>));\r\ntestCase.TestData.P2 = addpath(fullfile(pwd, <span class=\"string\">'path2'<\/span>));\r\n\r\nensureValidEnvironment(testCase);\r\ntestCase.TestData.WarnState = warning(<span class=\"string\">'off'<\/span>, <span class=\"string\">'MyProduct:MyFeature:MyWarnID'<\/span>);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> teardown(testCase)\r\nwarning(testCase.TestData.WarnState);\r\npath(testCase.TestData.P2);\r\npath(testCase.TestData.P1);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> testSomething(~)\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<\/pre><p>OK, the happy path here is fine, the teardown is executed in the reverse order of the setup and things look good. However, if any of the code in the setup method errors, has an assertion, or filters the test using an assumption we have a problem. What in the world is in that <b><tt>ensureValidEnvironment<\/tt><\/b> helper function anyway?<\/p><pre class=\"language-matlab\">\r\n<span class=\"keyword\">function<\/span> ensureValidEnvironment(testCase)\r\nassumeFalse(testCase, ismac, <span class=\"keyword\">...<\/span>\r\n    <span class=\"string\">'This feature is not supported on the Mac'<\/span>); \r\n\r\n<\/pre><p>It looks as though this test should be filtered on the Mac (which, as it turns out, happens to be my platform of choice). We may be in trouble. Let's see what happens.<\/p><pre class=\"codeinput\">runtests <span class=\"string\">NotExceptionSafeTest<\/span>;\r\n<\/pre><pre class=\"codeoutput\">Running NotExceptionSafeTest\r\n\r\n================================================================================\r\nNotExceptionSafeTest\/testSomething was filtered.\r\n    Test Diagnostic: This feature is not supported on the Mac\r\n================================================================================\r\n\r\n================================================================================\r\nError occurred in NotExceptionSafeTest\/testSomething and it did not run to completion.\r\n\r\n    --------------\r\n    Error Details:\r\n    --------------\r\n    Reference to non-existent field 'WarnState'.\r\n    \r\n    Error in NotExceptionSafeTest&gt;teardown (line 14)\r\n    warning(testCase.TestData.WarnState);\r\n    \r\n================================================================================\r\n.\r\nDone NotExceptionSafeTest\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                                Failed  Incomplete  Reason(s)\r\n    =================================================================================\r\n     NotExceptionSafeTest\/testSomething    X         X       Filtered by assumption.\r\n                                                             Errored.\r\n    \r\n<\/pre><p>The test is filtered correctly, but we run into problems when we try to teardown our fixtures and the test fails. This happens because the <b><tt>WarnState<\/tt><\/b> field was never added to the <b><tt>TestData<\/tt><\/b> and therefore when the framework executes teardown the first line errors. Worse, because this line errors we never restore the path to the starting value.<\/p><pre class=\"codeinput\">showTopThreePathFolders;\r\n<\/pre><pre class=\"codeoutput\">    '\/Users\/Shared\/blog\/addTeardown\/path2'\r\n    '\/Users\/Shared\/blog\/addTeardown\/path1'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n\r\n<\/pre><p>The <b><tt>path1<\/tt><\/b> and <b><tt>path2<\/tt><\/b> folders were never removed.<\/p><p>Again, using <b><tt>addTeardown<\/tt><\/b> simplifies and solves this problem:<\/p><pre class=\"language-matlab\"><span class=\"keyword\">function<\/span> tests = ExceptionSafeTest\r\ntests = functiontests(localfunctions);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> setup(testCase)\r\np1 = addpath(fullfile(pwd, <span class=\"string\">'path1'<\/span>));\r\naddTeardown(testCase, @path, p1);\r\n\r\np2 = addpath(fullfile(pwd, <span class=\"string\">'path2'<\/span>));\r\naddTeardown(testCase, @path, p2);\r\n\r\nensureValidEnvironment(testCase);\r\nwarnState = warning(<span class=\"string\">'off'<\/span>, <span class=\"string\">'MyProduct:MyFeature:MyWarnID'<\/span>);\r\naddTeardown(testCase, @warning, warnState);\r\n<span class=\"keyword\">end<\/span>\r\n\r\n<span class=\"keyword\">function<\/span> testSomething(~)\r\n<span class=\"keyword\">end<\/span>\r\n<\/pre><p>Now I like this for a few reasons:<\/p><div><ul><li>It is exception safe. If the test is filtered out by <b><tt>ensureValidEnvironment<\/tt><\/b> we are fine. Because the teardown has been added only for state that has actually changed, the framework does not try to restore the warning state at all but correctly restores the path.<\/li><li>It is shorter and simpler. No properties or stored values in <b><tt>TestData<\/tt><\/b> are needed because a single function contains all the data required.<\/li><li>We teardown as close as possible to the location in the code we setup which is very clean. We change the state at the same point in the code that we register that state to be cleaned up. A good rule of thumb is to restore the state exactly in the same location as you modify it.<\/li><\/ul><\/div><p>That's all fine and dandy, but does it work?<\/p><pre class=\"codeinput\">path(blogPostPath); <span class=\"comment\">% Restore to the original path<\/span>\r\nruntests <span class=\"string\">ExceptionSafeTest<\/span>;\r\n<\/pre><pre class=\"codeoutput\">Running ExceptionSafeTest\r\n\r\n================================================================================\r\nExceptionSafeTest\/testSomething was filtered.\r\n    Test Diagnostic: This feature is not supported on the Mac\r\n================================================================================\r\n.\r\nDone ExceptionSafeTest\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                             Failed  Incomplete  Reason(s)\r\n    ==============================================================================\r\n     ExceptionSafeTest\/testSomething              X       Filtered by assumption.\r\n    \r\n<\/pre><p>The test no longer errors unexpectedly, and the path is restored correctly:<\/p><pre class=\"codeinput\">showTopThreePathFolders;\r\n<\/pre><pre class=\"codeoutput\">    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath3'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath2'\r\n    '\/Users\/Shared\/blog\/addTeardown\/folderOnOriginalPath1'\r\n\r\n<\/pre><p><b><tt>testCase.addTeardown(@blogPost)<\/tt><\/b>;<\/p><p>So if you can't tell, I am a big advocate of <b><tt>addTeardown<\/tt><\/b>, and I use it almost exclusively in my own testing because I highly value robustness and am already familiar with the paradigm. Many people may not encounter these cases in their testing (or can forgive problems in failure scenarios). The <b><tt>TestMethodTeardown<\/tt><\/b> approach is more familiar from other xUnit implementations and is perfectly valid to use. That said, for the most robust, most production grade testing you should definitely consider <b><tt>addTeardown<\/tt><\/b>. What patterns and principles have you used to ensure safety and robustness of software fixtures? Let us know in the comments!<\/p><script language=\"JavaScript\"> <!-- \r\n    function grabCode_6d927ab3c00a4f2aa604ff7573eb9aa0() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='6d927ab3c00a4f2aa604ff7573eb9aa0 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 6d927ab3c00a4f2aa604ff7573eb9aa0';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2015 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_6d927ab3c00a4f2aa604ff7573eb9aa0()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; R2015a<br><\/p><\/div><!--\r\n6d927ab3c00a4f2aa604ff7573eb9aa0 ##### SOURCE BEGIN #####\r\n%% \r\n% I once faced a question from a friend who was perusing MATLAB\r\n% documentation while implementing a test suite  for some production\r\n% software and found an example test which used the\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcase.addteardown.html\r\n% *|addTeardown|*> method of the *|TestCase|* object. This person found\r\n% themselves wondering why they might want to use it to manage their test\r\n% fixture, citing that it was unclear where *|addTeardown|* fit with other\r\n% fixture related features like *|TestMethodTeardown|* and\r\n% *|TestClassTeardown|*. Alright, what is the deal with *|addTeardown|* and\r\n% why would we use it? This construct is actually not seen in most xUnit\r\n% paradigms so it may be unfamiliar to many. However, in fact\r\n% *|addTeardown|* is more robust and, once you get used to it, actually\r\n% less difficult than the other teardown routines. However its usage is\r\n% markedly different from standard xUnit paradigms, so\r\n% *|TestClassTeardown|* and *|TestMethodTeardown|* were included for\r\n% standard familiarity.\r\n% \r\n% Before we get too far into this, I am going to mysteriously add a few\r\n% dummy folders to the path, save the entire path contents, and take a\r\n% snapshot of these top three folders on the path using the following quick\r\n% helper function. These will be handy as we explore the topic.\r\n%\r\n% <include>showTopThreePathFolders.m<\/include>\r\n%\r\naddpath(fullfile(pwd, 'folderOnOriginalPath1'));\r\naddpath(fullfile(pwd, 'folderOnOriginalPath2'));\r\naddpath(fullfile(pwd, 'folderOnOriginalPath3'));\r\nblogPostPath = path;\r\nshowTopThreePathFolders;\r\n%% \r\n% Got it? Good, moving on.\r\n\r\n%%\r\n% *A House of Order*\r\n%\r\n% The first reason that *|addTeardown|* is more robust is that it gives an\r\n% order guarantee. The contract is that teardown actions occur in the\r\n% reverse order to which they were added. When different fixtures change\r\n% the same state, this prevents the teardown from executing out of order\r\n% and restoring to the wrong end state. Here is an example demonstrating\r\n% how this setup\/teardown can run in the wrong order without this feature:\r\n% \r\n% \r\n% <include>BadFixturesTest.m<\/include>\r\n%   \r\n% A couple things to note about this and the other examples in this post: \r\n%\r\n% * This example is contrived and is easy to work around, but this scenario\r\n% can be encountered in more subtle ways.\r\n% * The order of when setup\/teardown methods are executed is not\r\n% guaranteed. Even if they seem consistent now they can change. Therefore,\r\n% tweaking the order of the methods is not a good solution to this problem.\r\n% * There are other features in the framework that can be used instead of\r\n% this code, for example the\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.fixtures.pathfixture-class.html\r\n% *|PathFixture|*> and the\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.fixtures.suppressedwarningsfixture-class.html\r\n% *|SuppressedWarningsFixture|*>. Interestingly, however, using these\r\n% fixtures with\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcase.applyfixture.html\r\n% *|applyFixture|*> leverages *|addTeardown|*.\r\n% \r\n% Look at what can happen when the methods are executed. Again, the\r\n% order is not guaranteed so the order shown is possible in general:\r\nruntests BadFixturesTest;\r\nshowTopThreePathFolders;\r\n\r\n%% \r\n% The above test run demonstrates the following:\r\n%\r\n% # addPath1 REPLACE_WITH_DASH_DASH> P1 contains original path, path1 is now on the active\r\n% path.\r\n% # addPath2 REPLACE_WITH_DASH_DASH> P2 now contains the path with path1 on it, both path1\r\n% and path2 are now on the active path.\r\n% # restoreP1 REPLACE_WITH_DASH_DASH> The original path is now the active path.\r\n% # restoreP2 REPLACE_WITH_DASH_DASH> The path stored in addPath2 is now the active path. This\r\n% path still has path1 on it and is *NOT* the path from the start.\r\n% \r\n% As you can see, the path was clobbered because we had no guarantee of\r\n% order. One thought may be to use *|rmpath|* instead of *|path|* to restore\r\n% the state, but the problem with *|rmpath|* is that it might result in\r\n% removing something from the path that was on the path at the start. Using\r\n% the *|path|* function allows the test to ignore whether or not the folder\r\n% was on the path to start. Instead, we can solve this using *|addTeardown|*\r\n% (and the result is much smaller to boot):\r\n% \r\n% <include>GoodFixturesTest.m<\/include>\r\n% \r\npath(blogPostPath); % Restore to the original path\r\nruntests GoodFixturesTest;\r\nshowTopThreePathFolders;\r\n%%\r\n% Now it doesn't matter which setup method is executed first because\r\n% *|addTeardown|* executes in LIFO order. We have a guarantee that the\r\n% teardown is executed in the reverse order of setup.\r\n\r\n\r\n%%\r\n%\r\n% *Let's Make Exceptions*\r\n%\r\n% OK, now another way to enforce order of setup\/teardown is to write the\r\n% code as a single setup and teardown function and code the order directly.\r\n% This works but is not exception safe. We want to guarantee that the state\r\n% we modify in the test is correctly restored to the original state even in\r\n% the event of unexpected errors as well as other code flow interruptions\r\n% like assertion and assumption failures. However this is problematic when\r\n% there is a lot of fixture setup\/teardown in single methods. Observe the\r\n% following scenario, this time with a function-based test just to show\r\n% that *|addTeardown|* can be used there as well:\r\n% \r\n% <include>NotExceptionSafeTest.m<\/include>\r\n% \r\n% OK, the happy path here is fine, the teardown is executed in the reverse\r\n% order of the setup and things look good. However, if any of the code in\r\n% the setup method errors, has an assertion, or filters the test using an\r\n% assumption we have a problem. What in the world is in that\r\n% *|ensureValidEnvironment|* helper function anyway?\r\n%\r\n% <include>ensureValidEnvironment.m<\/include>\r\n%\r\n%%\r\n% It looks as though this test should be filtered on the Mac (which, as\r\n% it turns out, happens to be my platform of choice). We may be in trouble.\r\n% Let's see what happens.\r\nruntests NotExceptionSafeTest;\r\n\r\n%% \r\n% The test is filtered correctly, but we run into problems when we try to\r\n% teardown our fixtures and the test fails. This happens because the\r\n% *|WarnState|* field was never added to the *|TestData|* and therefore when\r\n% the framework executes teardown the first line errors. Worse, because\r\n% this line errors we never restore the path to the starting value.\r\nshowTopThreePathFolders;\r\n\r\n%%\r\n% The *|path1|* and *|path2|* folders were never removed.\r\n\r\n%%\r\n% Again, using *|addTeardown|* simplifies and solves this problem:\r\n% \r\n%   function tests = ExceptionSafeTest\r\n%   tests = functiontests(localfunctions);\r\n%   end\r\n% \r\n%   function setup(testCase)\r\n%   p1 = addpath(fullfile(pwd, 'path1'));\r\n%   addTeardown(testCase, @path, p1);\r\n% \r\n%   p2 = addpath(fullfile(pwd, 'path2'));\r\n%   addTeardown(testCase, @path, p2);\r\n% \r\n%   ensureValidEnvironment(testCase);\r\n%   warnState = warning('off', 'MyProduct:MyFeature:MyWarnID');\r\n%   addTeardown(testCase, @warning, warnState);\r\n%   end\r\n% \r\n%   function testSomething(~)\r\n%   end\r\n% \r\n% Now I like this for a few reasons: \r\n%\r\n% * It is exception safe. If the test is filtered out by\r\n% *|ensureValidEnvironment|* we are fine. Because the teardown has been added\r\n% only for state that has actually changed, the framework does not try to\r\n% restore the warning state at all but correctly restores the path.\r\n% * It is shorter and simpler. No properties or stored values in *|TestData|*\r\n% are needed because a single function contains all the data required.\r\n% * We teardown as close as possible to the location in the code we setup\r\n% which is very clean. We change the state at the same point in the code\r\n% that we register that state to be cleaned up. A good rule of thumb\r\n% is to restore the state exactly in the same location as you\r\n% modify it.\r\n%\r\n% That's all fine and dandy, but does it work?\r\npath(blogPostPath); % Restore to the original path\r\nruntests ExceptionSafeTest;\r\n\r\n%%\r\n% The test no longer errors unexpectedly, and the path is restored\r\n% correctly:\r\nshowTopThreePathFolders;\r\n\r\n%%\r\n%\r\n% *|testCase.addTeardown(@blogPost)|*;\r\n%\r\n% So if you can't tell, I am a big advocate of *|addTeardown|*, and I use it\r\n% almost exclusively in my own testing because I highly value robustness\r\n% and am already familiar with the paradigm. Many people may not encounter\r\n% these cases in their testing (or can forgive problems in failure\r\n% scenarios). The *|TestMethodTeardown|* approach is more familiar from other\r\n% xUnit implementations and is perfectly valid to use. That said, for the\r\n% most robust, most production grade testing you should definitely consider\r\n% *|addTeardown|*. What patterns and principles have you used to ensure\r\n% safety and robustness of software fixtures? Let us know in the comments!\r\n\r\n##### SOURCE END ##### 6d927ab3c00a4f2aa604ff7573eb9aa0\r\n-->","protected":false},"excerpt":{"rendered":"<p>I once faced a question from a friend who was perusing MATLAB documentation while implementing a test suite  for some production software and found an example test which used the addTeardown method... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/developer\/2015\/07\/27\/addteardown\/\">read more >><\/a><\/p>","protected":false},"author":90,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[7],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/218"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/users\/90"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/comments?post=218"}],"version-history":[{"count":6,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/218\/revisions"}],"predecessor-version":[{"id":229,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/posts\/218\/revisions\/229"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/media?parent=218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/categories?post=218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/developer\/wp-json\/wp\/v2\/tags?post=218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}