{"id":790,"date":"2013-10-15T13:36:56","date_gmt":"2013-10-15T18:36:56","guid":{"rendered":"https:\/\/blogs.mathworks.com\/loren\/?p=790"},"modified":"2016-08-04T09:00:54","modified_gmt":"2016-08-04T14:00:54","slug":"function-is-as-functiontests","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/loren\/2013\/10\/15\/function-is-as-functiontests\/","title":{"rendered":"Function is as functiontests"},"content":{"rendered":"<div class=\"content\"><!--introduction--><p>Today we have a post from guest blogger Andy Campbell to highlight a function-based approach for writing tests in MATLAB. We will be demonstrating how you can write simple functions that aid you in designing a physical system. This approach allows you to iterate quickly and improve your designs while protecting you from violating requirements met by earlier solutions.<\/p><p>With that, let's jump right in!<\/p><!--\/introduction--><h3>Contents<\/h3><div><ul><li><a href=\"#36234363-397c-4d20-ab68-a2746a5ea206\">Problem Statement<\/a><\/li><li><a href=\"#fffe4398-3e1d-4acf-98fe-5ba95144d772\">Enter <tt>functiontests<\/tt><\/a><\/li><li><a href=\"#99182b39-3a63-40a4-88f1-f22c44cfedb4\">What is <tt>functiontests<\/tt>?<\/a><\/li><li><a href=\"#5ae669f8-a00d-40e7-bfbf-43eebbdd1144\">What is <tt>localfunctions<\/tt>?<\/a><\/li><li><a href=\"#4816a096-bcd7-49e9-9896-4c8cf5f41584\">The First Test<\/a><\/li><li><a href=\"#e425df3a-d2ad-41b7-9a09-20e93e1d012a\">The First Design<\/a><\/li><li><a href=\"#a66a8f98-26df-4f91-849c-ddbee7f6772a\">The Second Test<\/a><\/li><li><a href=\"#3ed1c006-eaba-4d85-a931-d83726f5c632\">The Second Design<\/a><\/li><li><a href=\"#3c710089-6b82-4975-9005-f24cc473ce9b\">The Final Design<\/a><\/li><li><a href=\"#826979c3-542b-47c0-b5e5-d11738b77856\">Conclusion<\/a><\/li><\/ul><\/div><h4>Problem Statement<a name=\"36234363-397c-4d20-ab68-a2746a5ea206\"><\/a><\/h4><p>Suppose you are working on a design problem that is governed by a second order differential equation. The beauty of mathematics is that this actually has a wide range of applicability across many different areas. For our example, let's consider the design of a simple mass-spring-damper. Such a system may be used to model the suspension system on a car or truck. This is often represented using the following simplified model:<\/p><p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/2013\/MassSpringDamper.png\" alt=\"\"> <\/p><p>Where:<\/p><div><ul><li>$m$ = mass<\/li><li>$k$ = spring constant<\/li><li>$c$ = damping coefficient<\/li><li>$x$ = displacement from equilibrium<\/li><\/ul><\/div><p>Assuming there are no external forces, the equation of motion for this system is the solution to the second order differential equation $m\\ddot{x} + c\\dot{x} + kx = 0$. Following a similar approach to <a href=\"https:\/\/blogs.mathworks.com\/loren\/2010\/03\/25\/solving-ordinary-differential-equations\/\">an earlier blog post<\/a>, we can prepare this equation to be solved by one of MATLAB's ode solvers and write the following function used to evaluate our design:<\/p><pre class=\"codeinput\">type <span class=\"string\">simulateSystem<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction [x, t] = simulateSystem(design)\r\n\r\n% Design variables\r\nc = design.c;\r\nk = design.k;\r\n\r\n% Constant variables\r\nz0 = [-0.1; 0];  % Initial Position and Velocity\r\nm = 1500;        % Mass\r\n\r\nodefun = @(t,z) [0 1; -k\/m -c\/m]*z;\r\n[t, z] = ode45(odefun, [0, 3], z0);\r\n\r\n% The first column is the position (displacement from equilibrium)\r\nx = z(:, 1);\r\n<\/pre><p>In this function, $z$ is a vector representing the state of the system. It contains $x$ and its derivative $\\dot{x}$. Note that the initial conditions are that the mass has a position of -0.1 and its velocity is 0. This state may model the case where you are driving on the highway and hit a pothole. The initial position models the mass at the bottom of the pothole and the zero initial velocity models the suspension compressed, stopped, and about to recoil. For the sake of this example, let's say that the mass is constant at 1500 kg, which is reasonable for the load carried by a single wheel of a large light duty truck.<\/p><p>The design variables are the spring constant, $k$, and the damping coefficient, $c$. Initially they both have an arbitrary value of 1.<\/p><pre class=\"codeinput\">type <span class=\"string\">springDamperDesign0<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction design = springDamperDesign0\r\n% Design variables\r\ndesign.c = 1;          % Damping Coefficient (initial design)\r\ndesign.k = 1;          % Spring Constant (initial design)\r\n<\/pre><h4>Enter <tt>functiontests<\/tt><a name=\"fffe4398-3e1d-4acf-98fe-5ba95144d772\"><\/a><\/h4><p>If I were tasked to design a system like this I would undoubtedly look for some requirements of the design and ensure my design meets them all. However, a problem can surface if I implement a design that fits one requirement but then adjust the design to fit a second requirement that ends up breaking the first. For one or two requirements this is fairly easy to manage but it quickly becomes intractable with many requirements or if new requirements are added at a later time. Wouldn't it be great if we had some simple mechanism to capture the first requirement to ensure it is never violated? We do! Let's write a test!<\/p><p>An easy way to start writing your own test function is to use the <tt>functiontests<\/tt> function. The <tt>functiontests<\/tt> function can be called at the top of your test function file, and it allows you to create and group many related tests in the same file.<\/p><pre class=\"codeinput\">type <span class=\"string\">testTemplate<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction tests = testTemplate\r\n\r\ntests = functiontests(localfunctions);\r\n\r\nend\r\n<\/pre><p>Let's dissect what is happening here. By convention, the name of this file starts or ends with the word \"test\" (case insensitive) to help identify it as a function file that contains tests. All it needs to do is call the <tt>localfunctions<\/tt> function as input into the <tt>functiontests<\/tt> function, and then assign the result to the output of the main function.<\/p><h4>What is <tt>functiontests<\/tt>?<a name=\"99182b39-3a63-40a4-88f1-f22c44cfedb4\"><\/a><\/h4><p><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/functiontests.html\"><tt>functiontests<\/tt><\/a> is a MATLAB function (new in R2013b) that accepts a cell array of function handles to local functions within a MATLAB file. It then determines which of these local functions are tests based on naming convention and creates a <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testclass.html\"><tt>Test<\/tt><\/a> element for each of them. All local functions that start or end with the word \"test\" (case insensitive) are considered tests.<\/p><h4>What is <tt>localfunctions<\/tt>?<a name=\"5ae669f8-a00d-40e7-bfbf-43eebbdd1144\"><\/a><\/h4><p>The <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/localfunctions.html\"><tt>localfunctions<\/tt><\/a> function is another MATLAB function that is new in R2013b. This function takes no input arguments and simply returns a cell array of handles to all of the local functions within the calling file. When testing with <tt>functiontests<\/tt>, using <tt>localfunctions<\/tt> is highly recommended so that when you add additional test functions to the file their function handles do not need to be manually added to a cell array at the top of the file. Instead, <tt>localfunctions<\/tt> automatically discovers these new tests and adds them to the cell array of function handles passed to <tt>functiontests<\/tt>.<\/p><h4>The First Test<a name=\"4816a096-bcd7-49e9-9896-4c8cf5f41584\"><\/a><\/h4><p>With this basic framework in place we can write our first test. The first design requirement might be that the system returns to equilibrium within a certain amount of time, for example, 2 seconds. Let's write a test to see how we are doing against this requirement.<\/p><pre class=\"codeinput\">type <span class=\"string\">design0Test<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction tests = design0Test\r\n\r\ntests = functiontests(localfunctions);\r\n\r\nend\r\n\r\nfunction testSettlingTime(testCase) \r\n% Test that the system settles to within 0.001 of zero under 2 seconds.\r\n\r\n[position, time] = simulateSystem(springDamperDesign0); \r\n\r\npositionAfterSettling = position(time &gt; 2);\r\n\r\n% For this example, verify the first value after the settling time.\r\nverifyEqual(testCase, positionAfterSettling(1), 0, 'AbsTol', 0.001);\r\nend\r\n<\/pre><p>This test:<\/p><div><ol><li>Simulates the system using the specified design variables.<\/li><li>Captures the position after the required settling time.<\/li><li>Verifies that the position reaches 0 within +\/- 0.001 by the required time. Typically for a real test you would want to verify that all positions after the settling time are sufficiently close to zero, not just the point at the desired time.<\/li><\/ol><\/div><p>Note that this test function uses, and requires, a single <tt>testCase<\/tt> argument. This argument contains important test context and enables the use of a rich library of qualification functions such as <tt>verifyEqual<\/tt>.<\/p><p>What happens when we call this function?<\/p><pre class=\"codeinput\">test = design0Test;\r\ndisp(test)\r\n<\/pre><pre class=\"codeoutput\">  Test with properties:\r\n\r\n                  Name: 'design0Test\/testSettlingTime'\r\n    SharedTestFixtures: []\r\n<\/pre><p>You can see a single test was created from this function. To run this test, simply call the <tt>run<\/tt> function with this test as an argument.<\/p><pre class=\"codeinput\">run(test);\r\n<\/pre><pre class=\"codeoutput\">Running design0Test\r\n\r\n================================================================================\r\nVerification failed in design0Test\/testSettlingTime.\r\n\r\n    ---------------------\r\n    Framework Diagnostic:\r\n    ---------------------\r\n    verifyEqual failed.\r\n    --&gt; NumericComparator failed.\r\n        --&gt; The values are not equal using \"isequaln\".\r\n        --&gt; AbsoluteTolerance failed.\r\n            --&gt; The value was not within absolute tolerance.\r\n                \r\n                Tolerance Definition:\r\n                    abs(expected - actual) &lt;= tolerance\r\n                Tolerance Value:\r\n                         1.000000000000000e-03\r\n    \r\n    Actual Value:\r\n          -0.099863405108097\r\n    Expected Value:\r\n             0\r\n\r\n    ------------------\r\n    Stack Information:\r\n    ------------------\r\n    In H:\\Documents\\LOREN\\MyJob\\Art of MATLAB\\Andy Campbell\\informal test interface\\design0Test.m (testSettlingTime) at 15\r\n    In C:\\Program Files\\MATLAB\\R2013b\\toolbox\\matlab\\testframework\\+matlab\\+unittest\\FunctionTestCase.m (FunctionTestCase.test) at 90\r\n================================================================================\r\n.\r\nDone design0Test\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                          Failed  Incomplete  Reason(s)\r\n    ===========================================================================\r\n     design0Test\/testSettlingTime    X                 Failed by verification.\r\n    \r\n<\/pre><p>As you can see, the test failed and printed some information from <tt>verifyEqual<\/tt> to help diagnose the failure. Let's see how close we are graphically. Note that, for this example, in order to show the dynamics we need to show a much larger time span than our system should require. The <tt>simulateSystemOverLargeTime<\/tt> contains the same content as <tt>simulateSystem<\/tt> defined above but it has a different time span.<\/p><pre class=\"codeinput\">[x, t] = simulateSystemOverLargeTime(springDamperDesign0);\r\nplot(t, x);\r\ntitle(<span class=\"string\">'Undesigned Response'<\/span>);\r\nxlabel(<span class=\"string\">'Time'<\/span>);\r\nylabel(<span class=\"string\">'Position'<\/span>)\r\n<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/2013\/FunctionIsAsfunctiontests_01.png\" alt=\"\"> <p>This plot demonstrates how this design doesn't come close reaching equilibrium under 2 seconds (it is not even close after 1000 seconds). However, this is to be expected since we haven't designed anything yet.<\/p><h4>The First Design<a name=\"e425df3a-d2ad-41b7-9a09-20e93e1d012a\"><\/a><\/h4><p>Assume we come up with the following design:<\/p><pre class=\"codeinput\">type <span class=\"string\">springDamperDesign1<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction design = springDamperDesign1\r\ndesign.k = 5e6; % Spring Constant\r\ndesign.c = 1e4; % Damping Coefficient\r\n<\/pre><p>Does this design meet our criteria?<\/p><pre class=\"codeinput\">run(design1Test);\r\n<\/pre><pre class=\"codeoutput\">Running design1Test\r\n.\r\nDone design1Test\r\n__________\r\n\r\n<\/pre><p>Yes it does, we've passed! Let's take a look at our response:<\/p><pre class=\"codeinput\">[x, t] = simulateSystem(springDamperDesign1);\r\nplot(t, x)\r\ntitle(<span class=\"string\">'Design #1'<\/span>)\r\nxlabel(<span class=\"string\">'Time'<\/span>)\r\nylabel(<span class=\"string\">'Position'<\/span>)\r\n<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/2013\/FunctionIsAsfunctiontests_02.png\" alt=\"\"> <p>Great, we reach equilibrium rapidly enough. However, unfortunately it doesn't look like a comfortable ride in your brand new truck. It looks like all the vibration from hitting a pothole will shake the poor passengers' eyes right out of their sockets. In addition, the system has overshot the equilibrium point by 80% of the pothole depth (0.08 maximum position with a pothole depth of 0.1). We can do better. Let's write a test to prove it.<\/p><h4>The Second Test<a name=\"a66a8f98-26df-4f91-849c-ddbee7f6772a\"><\/a><\/h4><p>The missing requirement here could be a restriction on how far the position can overshoot the equilibrium point. We can now add a second test to our arsenal:<\/p><pre class=\"codeinput\">type <span class=\"string\">design1SecondTest<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction tests = design1SecondTest\r\n\r\ntests = functiontests(localfunctions); \r\n\r\nend\r\n\r\nfunction testSettlingTime(testCase) \r\n% Test that the system settles to within 0.001 of zero under 2 seconds.\r\n\r\n[position, time] = simulateSystem(springDamperDesign1); \r\n\r\npositionAfterSettling = position(time &gt; 2);\r\n\r\n% For this example, verify the first value after the settling time.\r\nverifyEqual(testCase, positionAfterSettling(1), 0, 'AbsTol', 0.001);\r\nend\r\n\r\nfunction testOvershoot(testCase)\r\n% Test to ensure that overshoot is less than 0.01\r\n\r\nposition = simulateSystem(springDamperDesign1);\r\novershoot = max(position);\r\n\r\nverifyLessThan(testCase, overshoot, 0.01);\r\nend\r\n<\/pre><p>This overshoot test uses <tt>verifyLessThan<\/tt> to ensure that the position never moves too far above the equilibrium point. We would expect this test to fail with our current design. First, we again create the <tt>Test<\/tt> array. Note that the second test was picked up automatically and the <tt>Test<\/tt> array now contains two elements instead of one.<\/p><pre class=\"codeinput\">tests = design1SecondTest;\r\ndisp(tests)\r\nrun(tests);\r\n<\/pre><pre class=\"codeoutput\">  1x2 Test array with properties:\r\n\r\n    Name\r\n    SharedTestFixtures\r\n\r\nRunning design1SecondTest\r\n.\r\n================================================================================\r\nVerification failed in design1SecondTest\/testOvershoot.\r\n\r\n    ---------------------\r\n    Framework Diagnostic:\r\n    ---------------------\r\n    verifyLessThan failed.\r\n    --&gt; The value must be less than the maximum value.\r\n    \r\n    Actual Value:\r\n           0.082943282378938\r\n    Maximum Value (Exclusive):\r\n           0.010000000000000\r\n\r\n    ------------------\r\n    Stack Information:\r\n    ------------------\r\n    In H:\\Documents\\LOREN\\MyJob\\Art of MATLAB\\Andy Campbell\\informal test interface\\design1SecondTest.m (testOvershoot) at 24\r\n    In C:\\Program Files\\MATLAB\\R2013b\\toolbox\\matlab\\testframework\\+matlab\\+unittest\\FunctionTestCase.m (FunctionTestCase.test) at 90\r\n================================================================================\r\n.\r\nDone design1SecondTest\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                             Failed  Incomplete  Reason(s)\r\n    ==============================================================================\r\n     design1SecondTest\/testOvershoot    X                 Failed by verification.\r\n    \r\n<\/pre><p>The overshoot test fails, and this is not a bad thing! Not only does it inform us that we are missing this requirement, it also serves as a sanity check to confirm that our tests are indeed doing what we think they are doing.<\/p><h4>The Second Design<a name=\"3ed1c006-eaba-4d85-a931-d83726f5c632\"><\/a><\/h4><p>Based on the response of the system, it seems our current design is underdamped, so why don't we just add some damping to prevent our overshoot and run the tests again.<\/p><pre class=\"codeinput\">type <span class=\"string\">springDamperDesign2<\/span>\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction design = springDamperDesign2\r\n\r\ndesign.k = 5e6;   % Spring Constant \r\ndesign.c = 2.5e6; % Increase the Damping Coefficient from 1e4\r\n<\/pre><p>Let's see how this does<\/p><pre class=\"codeinput\">[x, t] = simulateSystem(springDamperDesign2);\r\nplot(t, x)\r\ntitle(<span class=\"string\">'Design #2'<\/span>)\r\nxlabel(<span class=\"string\">'Time'<\/span>)\r\nylabel(<span class=\"string\">'Position'<\/span>)\r\n<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/2013\/FunctionIsAsfunctiontests_03.png\" alt=\"\"> <p>Looking at the response of this system it seems like a nice smooth ride that gently moves the truck back to equilibrium after the disturbance. Let's go with it! First let's just make sure it passes our tests.<\/p><pre class=\"codeinput\">run(design2Test);\r\n<\/pre><pre class=\"codeoutput\">Running design2Test\r\n\r\n================================================================================\r\nVerification failed in design2Test\/testSettlingTime.\r\n\r\n    ---------------------\r\n    Framework Diagnostic:\r\n    ---------------------\r\n    verifyEqual failed.\r\n    --&gt; NumericComparator failed.\r\n        --&gt; The values are not equal using \"isequaln\".\r\n        --&gt; AbsoluteTolerance failed.\r\n            --&gt; The value was not within absolute tolerance.\r\n                \r\n                Tolerance Definition:\r\n                    abs(expected - actual) &lt;= tolerance\r\n                Tolerance Value:\r\n                         1.000000000000000e-03\r\n    \r\n    Actual Value:\r\n          -0.001823826310161\r\n    Expected Value:\r\n             0\r\n\r\n    ------------------\r\n    Stack Information:\r\n    ------------------\r\n    In H:\\Documents\\LOREN\\MyJob\\Art of MATLAB\\Andy Campbell\\informal test interface\\design2Test.m (testSettlingTime) at 15\r\n    In C:\\Program Files\\MATLAB\\R2013b\\toolbox\\matlab\\testframework\\+matlab\\+unittest\\FunctionTestCase.m (FunctionTestCase.test) at 90\r\n================================================================================\r\n..\r\nDone design2Test\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                          Failed  Incomplete  Reason(s)\r\n    ===========================================================================\r\n     design2Test\/testSettlingTime    X                 Failed by verification.\r\n    \r\n<\/pre><p>Look at that! This design passes the overshoot test, but we have regressed and, once again, failed the settling time test. This is where we see the value in encoding these requirements through testing. There was some point after we secured the settling time requirement that we changed our focus to the overshoot requirement. It ended up being way too easy to evolve our design when tuning the overshoot and lose sight of the settling time. Focusing on overshoot, the latest design seems so reasonable, and since it is <i>close<\/i> to meeting the settling time requirement it was hard to notice through manual inspection. Obviously, close is not good enough! These tests are now protecting us from violating previously met requirements. These tests can be used to lockdown all past requirements, even when they are no longer in focus. Adding more tests can only make these requirements stricter or more complete.<\/p><p>This is especially important as the number of requirements for your system grows. You can become more protected by a larger and more comprehensive set of tests. You might imagine a case where you adjust the design in one way and 1 test out of 100 fails, but then you adjust the design in a different way and 75 tests out of 100 fail. When MATLAB is used to execute all of the tests in a single call you are able to gain insight into such cases, suggesting that the former design is likely closer to meeting your full set of requirements than the latter.<\/p><p>Since these tests use the standard MATLAB test framework they can be shared with colleagues and\/or customers who can easily run them and verify that they capture the requirements sufficiently. The tests act as an executable specification of the requirements, as well as the validation that these requirements are met.<\/p><h4>The Final Design<a name=\"3c710089-6b82-4975-9005-f24cc473ce9b\"><\/a><\/h4><p>For completeness, let's go ahead and make this test pass and then we can debrief. With a second order system we can design for critical damping and run the tests again.<\/p><pre class=\"codeinput\">type <span class=\"string\">springDamperDesign3.m<\/span>\r\nresult = run(design3Test);\r\n[x, t] = simulateSystem(springDamperDesign3);\r\nplot(t, x)\r\ntitle(<span class=\"string\">'Design #3'<\/span>)\r\nxlabel(<span class=\"string\">'Time'<\/span>)\r\nylabel(<span class=\"string\">'Position'<\/span>)\r\n<\/pre><pre class=\"codeoutput\">\r\nfunction design = springDamperDesign3\r\nm = 1500; % Need to know the mass to determine critical damping\r\n\r\ndesign.k = 5e6;                   % Spring Constant\r\ndesign.c = 2*m*sqrt(design.k\/m);  % Damping Coefficient to be critically damped\r\n\r\nRunning design3Test\r\n..\r\nDone design3Test\r\n__________\r\n\r\n<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/2013\/FunctionIsAsfunctiontests_04.png\" alt=\"\"> <p>The tests pass and the response looks as expected for critical damping. Now we can move forward improving our design with the knowledge that we are protected against failing to meet our first two requirements.<\/p><h4>Conclusion<a name=\"826979c3-542b-47c0-b5e5-d11738b77856\"><\/a><\/h4><p>Through a simplified example, we have demonstrated the power in using tests to validate engineering and scientific problems. All it took was writing a MATLAB function containing a couple of structured local functions. What's next? Well, for this design there might be any number of next steps, such as:<\/p><div><ol><li>Testing to ensure the spring constant is low enough to have a smoother, less jerky response.<\/li><li>Testing to limit the cost of the springs and\/or dampers (perhaps stiffer springs cost more money).<\/li><li>Testing to ensure the natural frequency of the system lies within a certain range.<\/li><li>Improving the model. In the real world an accurate model is likely much more complicated than a canonical second order system.<\/li><li>Testing the design against a payload. For example, if the mass needs to operate with cargo with additional mass within the range of 0 to 500 kg.<\/li><li>Testing the shape and characteristics of the frequency response.<\/li><\/ol><\/div><p>The tests we wrote today are testing only the results of a given design. Remember the underlying system does not need to be a mass-spring-damper at all. The design and\/or model can completely change, but these tests still are available to ensure that the response of the system meets the requirements.<\/p><p>The initial investment of writing a test up front yields lasting dividends and provides a safety net for analysis and design.  It is interesting that these activities are what we, as scientists and engineers, already do anyway. We always test our designs, we just don't always capture the process in the form of an executable test for future execution. This approach gets easier and easier to do the more you get the hang of it. Furthermore, any time invested in such tests up front quickly pays off in productivity gains by allowing safe, rapid design iterations.<\/p><p>This example only begins to scratch the surface of MATLAB testing capabilities. For more information, take a look at the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html\">documentation for the MATLAB Unit Test Framework<\/a>, which includes this <a title=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/write-function-based-unit-tests-.html (link no longer works)\">approach using test functions<\/a>. Let us know what you think <a href=\"https:\/\/blogs.mathworks.com\/loren\/?p=790#respond\">here<\/a>!<\/p><script language=\"JavaScript\"> <!-- \r\n    function grabCode_2e11c9fe08b34eeeba20ddaae1242c40() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='2e11c9fe08b34eeeba20ddaae1242c40 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 2e11c9fe08b34eeeba20ddaae1242c40';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2013 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_2e11c9fe08b34eeeba20ddaae1242c40()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; R2013b<br><\/p><p class=\"footer\"><br>\r\n      Published with MATLAB&reg; R2013b<br><\/p><\/div><!--\r\n2e11c9fe08b34eeeba20ddaae1242c40 ##### SOURCE BEGIN #####\r\n%% Function is as |functiontests|\r\n% Today we have a post from guest blogger Andy Campbell to highlight a\r\n% function-based approach for writing tests in MATLAB. We will be\r\n% demonstrating how you can write simple functions that aid you in\r\n% designing a physical system. This approach allows you to iterate quickly\r\n% and improve your designs while protecting you from violating requirements\r\n% met by earlier solutions.\r\n%\r\n% With that, let's jump right in!\r\n%\r\n%% Problem Statement\r\n% Suppose you are working on a design problem that is governed by a second\r\n% order differential equation. The beauty of mathematics is that this\r\n% actually has a wide range of applicability across many different areas.\r\n% For our example, let's consider the design of a simple\r\n% mass-spring-damper. Such a system may be used to model the suspension\r\n% system on a car or truck. This is often represented using the following\r\n% simplified model:\r\n% \r\n% <<MassSpringDamper.png>>\r\n%\r\n% Where:\r\n%\r\n% * $m$ = mass\r\n% * $k$ = spring constant\r\n% * $c$ = damping coefficient\r\n% * $x$ = displacement from equilibrium\r\n%\r\n% Assuming there are no external forces, the equation of motion for this\r\n% system is the solution to the second order differential equation\r\n% $m\\ddot{x} + c\\dot{x} + kx = 0$. Following a similar approach to\r\n% <https:\/\/blogs.mathworks.com\/loren\/2010\/03\/25\/solving-ordinary-differential-equations\/\r\n% an earlier blog post>, we can prepare this equation to be solved by one\r\n% of MATLAB's ode solvers and write the following function used to\r\n% evaluate our design:\r\ntype simulateSystem\r\n\r\n%% \r\n% In this function, $z$ is a vector representing the state of the system.\r\n% It contains $x$ and its derivative $\\dot{x}$. Note that the initial\r\n% conditions are that the mass has a position of -0.1 and its velocity is\r\n% 0. This state may model the case where you are driving on the highway and\r\n% hit a pothole. The initial position models the mass at the bottom of the\r\n% pothole and the zero initial velocity models the suspension compressed,\r\n% stopped, and about to recoil. For the sake of this example, let's say\r\n% that the mass is constant at 1500 kg, which is reasonable for the load\r\n% carried by a single wheel of a large light duty truck.\r\n%\r\n% The design variables are the spring constant, $k$, and the damping\r\n% coefficient, $c$. Initially they both have an arbitrary value of 1.\r\ntype springDamperDesign0\r\n\r\n%% Enter |functiontests|\r\n% If I were tasked to design a system like this I would undoubtedly look\r\n% for some requirements of the design and ensure my design meets them all.\r\n% However, a problem can surface if I implement a design that fits one\r\n% requirement but then adjust the design to fit a second requirement that\r\n% ends up breaking the first. For one or two requirements this is fairly\r\n% easy to manage but it quickly becomes intractable with many requirements\r\n% or if new requirements are added at a later time. Wouldn't it be great if\r\n% we had some simple mechanism to capture the first requirement to ensure it\r\n% is never violated? We do! Let's write a test!\r\n%\r\n% An easy way to start writing your own test function is to use the\r\n% |functiontests| function. The |functiontests| function can be called at\r\n% the top of your test function file, and it allows you to create and group\r\n% many related tests in the same file.\r\ntype testTemplate\r\n\r\n\r\n%% \r\n% Let's dissect what is happening here. By convention, the name of this\r\n% file starts or ends with the word \"test\" (case insensitive) to help\r\n% identify it as a function file that contains tests. All it needs to do is\r\n% call the |localfunctions| function as input into the |functiontests|\r\n% function, and then assign the result to the output of the main function.\r\n\r\n%% What is |functiontests|? \r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/functiontests.html\r\n% |functiontests|> is a MATLAB function (new in R2013b) that accepts a\r\n% cell array of function handles to local functions within a MATLAB file.\r\n% It then determines which of these local functions are tests based on\r\n% naming convention and creates a\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testclass.html\r\n% |Test|> element for each of them. All local functions that start or end\r\n% with the word \"test\" (case insensitive) are considered tests.\r\n\r\n\r\n%% What is |localfunctions|?\r\n% The <https:\/\/www.mathworks.com\/help\/matlab\/ref\/localfunctions.html\r\n% |localfunctions|> function is another MATLAB function that is new in\r\n% R2013b. This function takes no input arguments and simply returns a cell\r\n% array of handles to all of the local functions within the calling file.\r\n% When testing with |functiontests|, using |localfunctions| is highly\r\n% recommended so that when you add additional test functions to the file\r\n% their function handles do not need to be manually added to a cell array\r\n% at the top of the file. Instead, |localfunctions| automatically discovers\r\n% these new tests and adds them to the cell array of function handles\r\n% passed to |functiontests|.\r\n\r\n%% The First Test\r\n% With this basic framework in place we can write our first test. The first\r\n% design requirement might be that the system returns to equilibrium within\r\n% a certain amount of time, for example, 2 seconds. Let's write a test to\r\n% see how we are doing against this requirement.\r\ntype design0Test\r\n\r\n%%\r\n% This test:\r\n%\r\n% # Simulates the system using the specified design variables.\r\n% # Captures the position after the required settling time.\r\n% # Verifies that the position reaches 0 within +\/- 0.001 by the\r\n% required time. Typically for a real test you would want to verify that\r\n% all positions after the settling time are sufficiently close to zero, not\r\n% just the point at the desired time.\r\n% \r\n% \r\n% Note that this test function uses, and requires, a single |testCase|\r\n% argument. This argument contains important test context and enables the\r\n% use of a\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/select-qualifications.html\r\n% rich library of qualification functions> such as |verifyEqual|.\r\n% \r\n% What happens when we call this function?\r\ntest = design0Test;\r\ndisp(test)\r\n\r\n%%\r\n% You can see a single test was created from this function. To run this\r\n% test, simply call the |run| function with this test as an\r\n% argument.\r\nrun(test);\r\n\r\n\r\n%%\r\n% As you can see, the test failed and printed some information from\r\n% |verifyEqual| to help diagnose the failure. Let's see how close we are\r\n% graphically. Note that, for this example, in order to show the dynamics\r\n% we need to show a much larger time span than our system should require.\r\n% The |simulateSystemOverLargeTime| contains the same content as\r\n% |simulateSystem| defined above but it has a different time span.\r\n[x, t] = simulateSystemOverLargeTime(springDamperDesign0);\r\nplot(t, x);\r\ntitle('Undesigned Response');\r\nxlabel('Time');\r\nylabel('Position')\r\n\r\n%%\r\n% This plot demonstrates how this design doesn't come close reaching\r\n% equilibrium under 2 seconds (it is not even close after 1000 seconds).\r\n% However, this is to be expected since we haven't designed anything yet.\r\n\r\n%% The First Design\r\n% Assume we come up with the following design:\r\ntype springDamperDesign1\r\n\r\n%%\r\n% Does this design meet our criteria?\r\nrun(design1Test);\r\n\r\n%%\r\n% Yes it does, we've passed! Let's take a look at our response:\r\n[x, t] = simulateSystem(springDamperDesign1);\r\nplot(t, x)\r\ntitle('Design #1')\r\nxlabel('Time')\r\nylabel('Position')\r\n\r\n%%\r\n% Great, we reach equilibrium rapidly enough. However, unfortunately it\r\n% doesn't look like a comfortable ride in your brand new truck. It looks\r\n% like all the vibration from hitting a pothole will shake the poor\r\n% passengers' eyes right out of their sockets. In addition, the system has\r\n% overshot the equilibrium point by 80% of the pothole depth (0.08 maximum\r\n% position with a pothole depth of 0.1). We can do better. Let's write a\r\n% test to prove it.\r\n\r\n%% The Second Test\r\n% The missing requirement here could be a restriction on how far the\r\n% position can overshoot the equilibrium point. We can now add a second test\r\n% to our arsenal:\r\ntype design1SecondTest\r\n\r\n%%\r\n% This overshoot test uses |verifyLessThan| to ensure that the position never\r\n% moves too far above the equilibrium point. We would expect this test to\r\n% fail with our current design. First, we again create the |Test| array.\r\n% Note that the second test was picked up automatically and the |Test| array\r\n% now contains two elements instead of one.\r\ntests = design1SecondTest;\r\ndisp(tests)\r\nrun(tests);\r\n\r\n%%\r\n% The overshoot test fails, and this is not a bad thing! Not only does it\r\n% inform us that we are missing this requirement, it also serves as a\r\n% sanity check to confirm that our tests are indeed doing what we think\r\n% they are doing.\r\n%\r\n%% The Second Design\r\n% Based on the response of the system, it seems our current design is\r\n% underdamped, so why don't we just add some damping to prevent our\r\n% overshoot and run the tests again.\r\ntype springDamperDesign2\r\n\r\n%%\r\n% Let's see how this does\r\n[x, t] = simulateSystem(springDamperDesign2);\r\nplot(t, x)\r\ntitle('Design #2')\r\nxlabel('Time')\r\nylabel('Position')\r\n\r\n%%\r\n% Looking at the response of this system it seems like a nice smooth ride\r\n% that gently moves the truck back to equilibrium after the disturbance.\r\n% Let's go with it! First let's just make sure it passes our\r\n% tests.\r\nrun(design2Test);\r\n\r\n%%\r\n% Look at that! This design passes the overshoot test, but we have\r\n% regressed and, once again, failed the settling time test. This is where\r\n% we see the value in encoding these requirements through testing. There\r\n% was some point after we secured the settling time requirement that we\r\n% changed our focus to the overshoot requirement. It ended up being way too\r\n% easy to evolve our design when tuning the overshoot and lose sight of the\r\n% settling time. Focusing on overshoot, the latest design seems so\r\n% reasonable, and since it is _close_ to meeting the settling time\r\n% requirement it was hard to notice through manual inspection. Obviously,\r\n% close is not good enough! These tests are now protecting us from\r\n% violating previously met requirements. These tests can be used to\r\n% lockdown all past requirements, even when they are no longer in focus.\r\n% Adding more tests can only make these requirements stricter or more\r\n% complete.\r\n%\r\n% This is especially important as the number of requirements for your\r\n% system grows. You can become more protected by a larger and more\r\n% comprehensive set of tests. You might imagine a case where you adjust the\r\n% design in one way and 1 test out of 100 fails, but then you adjust the\r\n% design in a different way and 75 tests out of 100 fail. When MATLAB is\r\n% used to execute all of the tests in a single call you are able to gain\r\n% insight into such cases, suggesting that the former design is likely\r\n% closer to meeting your full set of requirements than the latter.\r\n%\r\n% Since these tests use the standard MATLAB test framework they can be\r\n% shared with colleagues and\/or customers who can easily run them and\r\n% verify that they capture the requirements sufficiently. The tests act as\r\n% an executable specification of the requirements, as well as the\r\n% validation that these requirements are met.\r\n%\r\n%% The Final Design\r\n% For completeness, let's go ahead and make this test pass and then we can\r\n% debrief. With a second order system we can design for critical damping\r\n% and run the tests again.\r\ntype springDamperDesign3.m\r\nresult = run(design3Test);\r\n[x, t] = simulateSystem(springDamperDesign3);\r\nplot(t, x)\r\ntitle('Design #3')\r\nxlabel('Time')\r\nylabel('Position')\r\n\r\n\r\n%%\r\n% The tests pass and the response looks as expected for critical damping.\r\n% Now we can move forward improving our design with the knowledge that we\r\n% are protected against failing to meet our first two requirements.\r\n\r\n%% Conclusion\r\n% Through a simplified example, we have demonstrated the power in using\r\n% tests to validate engineering and scientific problems. All it took was\r\n% writing a MATLAB function containing a couple of structured local\r\n% functions. What's next? Well, for this design there might be any number\r\n% of next steps, such as:\r\n%\r\n% # Testing to ensure the spring constant is low enough to have a\r\n% smoother, less jerky response.\r\n% # Testing to limit the cost of the springs and\/or dampers\r\n% (perhaps stiffer springs cost more money).\r\n% # Testing to ensure the natural frequency of the system lies within a\r\n% certain range.\r\n% # Improving the model. In the real world an accurate model is likely much\r\n% more complicated than a canonical second order system.\r\n% # Testing the design against a payload. For example, if the mass needs to\r\n% operate with cargo with additional mass within the range of 0 to 500 kg.\r\n% # Testing the shape and characteristics of the frequency response.\r\n% \r\n% The tests we wrote today are testing only the results of a given design.\r\n% Remember the underlying system does not need to be a mass-spring-damper\r\n% at all. The design and\/or model can completely change, but these tests\r\n% still are available to ensure that the response of the system meets the\r\n% requirements.\r\n%\r\n% The initial investment of writing a test up front yields lasting\r\n% dividends and provides a safety net for analysis and design.  It is\r\n% interesting that these activities are what we, as scientists and\r\n% engineers, already do anyway. We always test our designs, we just don't\r\n% always capture the process in the form of an executable test for future\r\n% execution. This approach gets easier and easier to do the more you get\r\n% the hang of it. Furthermore, any time invested in such tests up front\r\n% quickly pays off in productivity gains by allowing safe, rapid design\r\n% iterations.\r\n%\r\n% This example only begins to scratch the surface of MATLAB testing\r\n% capabilities. For more information, take a look at the\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html\r\n% documentation for the MATLAB Unit Test Framework>, which includes this\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/write-function-based-unit-tests-.html\r\n% approach using test functions>. Let us know what you think\r\n% <https:\/\/blogs.mathworks.com\/loren\/?p=790#respond here>!\r\n\r\n\r\n##### SOURCE END ##### 2e11c9fe08b34eeeba20ddaae1242c40\r\n-->","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/2013\/FunctionIsAsfunctiontests_04.png\" onError=\"this.style.display ='none';\" \/><\/div><!--introduction--><p>Today we have a post from guest blogger Andy Campbell to highlight a function-based approach for writing tests in MATLAB. We will be demonstrating how you can write simple functions that aid you in designing a physical system. This approach allows you to iterate quickly and improve your designs while protecting you from violating requirements met by earlier solutions.... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/loren\/2013\/10\/15\/function-is-as-functiontests\/\">read more >><\/a><\/p>","protected":false},"author":39,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[39,6,13],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/790"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/comments?post=790"}],"version-history":[{"count":9,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/790\/revisions"}],"predecessor-version":[{"id":1959,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/790\/revisions\/1959"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/media?parent=790"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/categories?post=790"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/tags?post=790"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}