{"id":649,"date":"2013-03-14T10:18:23","date_gmt":"2013-03-14T15:18:23","guid":{"rendered":"https:\/\/blogs.mathworks.com\/loren\/?p=649"},"modified":"2016-07-31T20:51:23","modified_gmt":"2016-08-01T01:51:23","slug":"using-the-matlab-unit-testing-infrastructure-for-grading-assignments","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/loren\/2013\/03\/14\/using-the-matlab-unit-testing-infrastructure-for-grading-assignments\/","title":{"rendered":"Using the MATLAB Unit Testing Infrastructure for Grading Assignments"},"content":{"rendered":"<div class=\"content\">\n<p><!--introduction-->Steven Lord, Andy Campbell, and David Hruska are members of the Quality Engineering group at MathWorks who are guest blogging today to introduce a new feature in R2013a, the MATLAB unit testing infrastructure. There are several submissions on the MATLAB Central File Exchange related to unit testing of <tt>MATLAB<\/tt> code. Blogger Steve Eddins wrote one highly rated example back in 2009. In release R2013a, MathWorks included in <tt>MATLAB<\/tt> itself a <tt>MATLAB<\/tt> implementation of the industry-standard xUnit testing framework.<\/p>\n<p>If you're not a software developer, you may be wondering if this feature will be of any use to you. In this post, we will describe one way someone who may not consider themselves a software developer may be able to take advantage of this framework using the example of a professor grading students' homework submissions. That's not to say that the developers in the audience should move on to the next post; you can use these tools to test your own code just like a professor can use them to test code written by his or her students.<\/p>\n<p>There is a great deal of functionality in this feature that we will not show here. For more information we refer you to the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html\">MATLAB Unit Testing Framework<\/a> documentation.<\/p>\n<p><!--\/introduction--><\/p>\n<h3>Contents<\/h3>\n<div>\n<ul>\n<li><a href=\"#454b2d47-cbbb-48bf-b2ba-50c3891d7f7a\">Background<\/a><\/li>\n<li><a href=\"#a7a02684-3047-424b-a927-3b218f015807\">Problem Statement<\/a><\/li>\n<li><a href=\"#eeac7ca5-eb4b-4028-9a87-c969cd27c754\">Basic Unit Test<\/a><\/li>\n<li><a href=\"#23717901-ce74-47ed-9564-08b52b69c2a4\">Running a Test<\/a><\/li>\n<li><a href=\"#8cf80ad8-6ac6-4606-951c-441a8ba5bcd7\">Test that F(0) Equals 1<\/a><\/li>\n<li><a href=\"#15df7917-0f94-4a3f-b915-552df373e33f\">Test that F(pi) Throws an Error<\/a><\/li>\n<li><a href=\"#3ad5bde5-93a3-4204-b976-8a60edad5fc6\">Basic Test for Students, Advanced Tests for Instructor<\/a><\/li>\n<li><a href=\"#b8d2ed9f-3356-4308-bc9b-b269f74b93bd\">Conclusion<\/a><\/li>\n<\/ul>\n<\/div>\n<h4>Background<a name=\"454b2d47-cbbb-48bf-b2ba-50c3891d7f7a\"><\/a><\/h4>\n<p>In order to use this feature, you should be aware of how to define simple <tt>MATLAB<\/tt> classes in <i>classdef<\/i> files, how to define a class that inherits from another, and how to specify attributes for methods and properties of those classes. The <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/object-oriented-programming.html\">object-oriented programming<\/a> documentation describes these capabilities.<\/p>\n<h4>Problem Statement<a name=\"a7a02684-3047-424b-a927-3b218f015807\"><\/a><\/h4>\n<p>As a professor in an introductory programming class, you want your students to write a program to compute Fibonacci numbers. The exact problem statement you give the students is:<\/p>\n<pre>Create a function \"fib\" that accepts a nonnegative integer n and returns\r\nthe nth Fibonacci number. The Fibonacci numbers are generated by this\r\nrelationship:<\/pre>\n<pre>F(0) = 1\r\nF(1) = 1\r\nF(n) = F(n-1) + F(n-2) for integer n &gt; 1<\/pre>\n<pre>Your function should throw an error if n is not a nonnegative integer.<\/pre>\n<h4>Basic Unit Test<a name=\"eeac7ca5-eb4b-4028-9a87-c969cd27c754\"><\/a><\/h4>\n<p>The most basic <tt>MATLAB<\/tt> unit test is a <tt>MATLAB<\/tt> <i>classdef<\/i> class file that inherits from the <tt>matlab.unittest.TestCase<\/tt> class. Throughout the rest of this post we will add additional pieces to this basic framework to increase the capability of this test and will change its name to reflect its increased functionality.<\/p>\n<pre class=\"codeinput\">dbtype <span class=\"string\">basicTest.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     classdef basicTest &lt; matlab.unittest.TestCase\r\n2         \r\n3     end\r\n<\/pre>\n<pre class=\"codeinput\">test = basicTest\r\n<\/pre>\n<pre class=\"codeoutput\">test = \r\n  basicTest with no properties.\r\n<\/pre>\n<h4>Running a Test<a name=\"23717901-ce74-47ed-9564-08b52b69c2a4\"><\/a><\/h4>\n<p>To run the test, we can simply pass <tt>test<\/tt> to the <tt>run<\/tt> function. There are more advanced ways that make it easier to run a group of tests, but for our purposes (checking one student's answer at a time) this will be sufficient. When you move to checking multiple students' answers at a time, you can use <tt>run<\/tt> inside a <tt>for<\/tt> loop.<\/p>\n<p>Since <tt>basicTest<\/tt> doesn't actually validate the output from the student's function, it doesn't take very long to execute.<\/p>\n<pre class=\"codeinput\">results = run(test)\r\n<\/pre>\n<pre class=\"codeoutput\">results = \r\n  0x0 TestResult array with properties:\r\n\r\n    Name\r\n    Passed\r\n    Failed\r\n    Incomplete\r\n    Duration\r\nTotals:\r\n   0 Passed, 0 Failed, 0 Incomplete.\r\n   0 seconds testing time.\r\n<\/pre>\n<p>Let's say that a student named Thomas submitted a function <tt>fib.m<\/tt> as his solution to this assignment. Thomas's code is stored in a sub-folder named <tt>thomas<\/tt>. To set up our test to check Thomas's answer, we add the folder holding his code to the path.<\/p>\n<pre class=\"codeinput\">addpath(<span class=\"string\">'thomas'<\/span>);\r\ndbtype <span class=\"string\">fib.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     function y = fib(n)\r\n2     if n &lt;= 1\r\n3         y = 1;\r\n4     else\r\n5         y = fib(n-1)+fib(n-2);\r\n6     end\r\n\r\n<\/pre>\n<h4>Test that F(0) Equals 1<a name=\"8cf80ad8-6ac6-4606-951c-441a8ba5bcd7\"><\/a><\/h4>\n<p>The <tt>basicTest<\/tt> is a valid test class, and we can run it, but it doesn't actually perform any validation of the student's test file. The methods that will perform that validation need to be written in a <i>methods<\/i> block that has the attribute <tt>Test<\/tt> specified.<\/p>\n<p>The <tt>matlab.unittest.TestCase<\/tt> class includes qualification methods that you can use to test various qualities of the results returned by the student files. The qualification method that you will likely use most frequently is the <tt>verifyEqual<\/tt> method, which passes if the two values you pass into it are equal and reports a test failure if they are not.<\/p>\n<p>The <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcaseclass.html\">documentation<\/a> for the <tt>matlab.unittest.TestCase<\/tt> class lists many other qualification methods that you can use to perform other types of validation, including testing the data type and size of the results; matching a string result to an expected string; testing that a given section of code throws a specific errors or issues a specific warning; and many more.<\/p>\n<p>This simple test builds upon <tt>generalTest<\/tt> by adding a test method that checks that the student's function returns the value 1 when called with the input 0.<\/p>\n<pre class=\"codeinput\">dbtype <span class=\"string\">simpleTest.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     classdef simpleTest &lt; matlab.unittest.TestCase\r\n2         methods(Test)\r\n3             function fibonacciOfZeroShouldBeOne(testCase)\r\n4                 % Evaluate the student's function for n = 0\r\n5                 result = fib(0);\r\n6                 testCase.verifyEqual(result, 1);\r\n7             end\r\n8         end\r\n9     end\r\n<\/pre>\n<p>Thomas's solution to the assignment satisfies this basic check. We can use the results returned from <tt>run<\/tt> to display the percentage of the tests that pass.<\/p>\n<pre class=\"codeinput\">results = run(simpleTest)\r\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\r\ndisp([num2str(percentPassed), <span class=\"string\">'% Passed.'<\/span>]);\r\n<\/pre>\n<pre class=\"codeoutput\">Running simpleTest\r\n.\r\nDone simpleTest\r\n__________\r\n\r\nresults = \r\n  TestResult with properties:\r\n\r\n          Name: 'simpleTest\/fibonacciOfZeroShouldBeOne'\r\n        Passed: 1\r\n        Failed: 0\r\n    Incomplete: 0\r\n      Duration: 0.0112\r\nTotals:\r\n   1 Passed, 0 Failed, 0 Incomplete.\r\n   0.011168 seconds testing time.\r\n100% Passed.\r\n<\/pre>\n<h4>Test that F(pi) Throws an Error<a name=\"15df7917-0f94-4a3f-b915-552df373e33f\"><\/a><\/h4>\n<p>Now that we have a basic positive test in place we can add in a test that checks the behavior of the student's function when passed a non-integer value (like <tt>n = pi<\/tt>) as input. The assignment stated that when called with a non-integer value, the student's function should error. Since the assignment doesn't require a specific error to be thrown, the test passes as long as <tt>fib(pi)<\/tt> throws any exception.<\/p>\n<pre class=\"codeinput\">dbtype <span class=\"string\">errorCaseTest.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     classdef errorCaseTest &lt; matlab.unittest.TestCase\r\n2         methods(Test)\r\n3             function fibonacciOfZeroShouldBeOne(testCase)\r\n4                 % Evaluate the student's function for n = 0\r\n5                 result = fib(0);\r\n6                 testCase.verifyEqual(result, 1);\r\n7             end\r\n8             function fibonacciOfNonintegerShouldError(testCase)\r\n9                 testCase.verifyError(@()fib(pi), ?MException);\r\n10            end\r\n11        end\r\n12    end\r\n<\/pre>\n<p>Thomas forgot to include a check for a non-integer valued input in his function, so our test should indicate that by reporting a failure.<\/p>\n<pre class=\"codeinput\">results = run(errorCaseTest)\r\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\r\ndisp([num2str(percentPassed), <span class=\"string\">'% Passed.'<\/span>]);\r\n<\/pre>\n<pre class=\"codeoutput\">Running errorCaseTest\r\n.\r\n================================================================================\r\nVerification failed in errorCaseTest\/fibonacciOfNonintegerShouldError.\r\n\r\n    ---------------------\r\n    Framework Diagnostic:\r\n    ---------------------\r\n    verifyError failed.\r\n    --&gt; The function did not throw any exception.\r\n        \r\n        Expected Exception Type:\r\n            MException\r\n    \r\n    Evaluated Function:\r\n            @()fib(pi)\r\n\r\n    ------------------\r\n    Stack Information:\r\n    ------------------\r\n    In C:\\Program Files\\MATLAB\\R2013a\\toolbox\\matlab\\testframework\\+matlab\\+unittest\\+qualifications\\Verifiable.m (Verifiable.verifyError) at 637\r\n    In H:\\Documents\\LOREN\\MyJob\\Art of MATLAB\\errorCaseTest.m (errorCaseTest.fibonacciOfNonintegerShouldError) at 9\r\n================================================================================\r\n.\r\nDone errorCaseTest\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                                            Failed  Incomplete  Reason(s)\r\n    =============================================================================================\r\n     errorCaseTest\/fibonacciOfNonintegerShouldError    X                 Failed by verification.\r\n    \r\nresults = \r\n  1x2 TestResult array with properties:\r\n\r\n    Name\r\n    Passed\r\n    Failed\r\n    Incomplete\r\n    Duration\r\nTotals:\r\n   1 Passed, 1 Failed, 0 Incomplete.\r\n   0.026224 seconds testing time.\r\n50% Passed.\r\n<\/pre>\n<p>Another student, Benjamin, checked for a non-integer value in his code as you can see on line 2.<\/p>\n<pre class=\"codeinput\">rmpath(<span class=\"string\">'thomas'<\/span>);\r\naddpath(<span class=\"string\">'benjamin'<\/span>);\r\ndbtype <span class=\"string\">fib.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     function y = fib(n)\r\n2     if (n ~= round(n)) || n &lt; 0\r\n3         error('N is not an integer!');\r\n4     elseif n == 0 || n == 1\r\n5         y = 1;\r\n6     else\r\n7         y = fib(n-1)+fib(n-2);\r\n8     end\r\n<\/pre>\n<p>Benjamin's code passed both the test implemented in the <tt>fibonacciOfZeroShouldBeOne<\/tt> method (which we copied into <tt>errorCaseTest<\/tt> from <tt>simpleTest<\/tt>) and the new test case implemented in the <tt>fibonacciOfNonintegerShouldError<\/tt> method.<\/p>\n<pre class=\"codeinput\">results = run(errorCaseTest)\r\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\r\ndisp([num2str(percentPassed), <span class=\"string\">'% Passed.'<\/span>]);\r\n<\/pre>\n<pre class=\"codeoutput\">Running errorCaseTest\r\n..\r\nDone errorCaseTest\r\n__________\r\n\r\nresults = \r\n  1x2 TestResult array with properties:\r\n\r\n    Name\r\n    Passed\r\n    Failed\r\n    Incomplete\r\n    Duration\r\nTotals:\r\n   2 Passed, 0 Failed, 0 Incomplete.\r\n   0.010132 seconds testing time.\r\n100% Passed.\r\n<\/pre>\n<h4>Basic Test for Students, Advanced Tests for Instructor<a name=\"3ad5bde5-93a3-4204-b976-8a60edad5fc6\"><\/a><\/h4>\n<p>The problem statement given earlier in this post is a plain text description of the homework assignment we assigned to the students. We can also state the problem for the students in code (if they're using release R2013a or later) by giving them a test file they can run just like <tt>simpleTest<\/tt> or <tt>errorCaseTest<\/tt>. They can directly use this \"requirement test\" to ensure their functions satisfy the requirements of the assignment.<\/p>\n<pre class=\"codeinput\">dbtype <span class=\"string\">studentTest.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     classdef studentTest &lt; matlab.unittest.TestCase\r\n2         methods(Test)\r\n3             function fibonacciOfZeroShouldBeOne(testCase)\r\n4                 % Evaluate the student's function for n = 0\r\n5                 result = fib(0);\r\n6                 testCase.verifyEqual(result, 1);\r\n7             end\r\n8             function fibonacciOfNonintegerShouldError(testCase)\r\n9                 testCase.verifyError(@()fib(pi), ?MException);\r\n10            end\r\n11        end\r\n12    end\r\n<\/pre>\n<p>In order for the student's code to pass the assignment, it will need to pass the test cases given in the <tt>studentTest<\/tt> unit test. However, we don't want to use <tt>studentTest<\/tt> as the <i>only<\/i> check of the student's code. If we did, the student could write their function to cover only the test cases in the student test file.<\/p>\n<p>We could solve this problem by having two separate test files, one containing the student test cases and one containing additional test cases the instructor uses in the grading process. Can we avoid having to run both test files manually or duplicating the code from the student test cases in the instructor test? Yes!<\/p>\n<p>To do so, we write an instructor test file to incorporate, through inheritance, the student test file. We can then add additional test cases to the instructor test file. When we run this test it should run three test cases; two inherited from <tt>studentTest<\/tt>, <tt>fibonacciOfZeroShouldBeOne<\/tt> and <tt>fibonacciOfNonintegerShouldError<\/tt>, and one from <tt>instructorTest<\/tt> itself, <tt>fibonacciOf5<\/tt>.<\/p>\n<pre class=\"codeinput\">dbtype <span class=\"string\">instructorTest.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     classdef instructorTest &lt; studentTest\r\n2         % Because the student test file is a matlab.unittest.TestCase and\r\n3         % instructorTest inherits from it, instructorTest is also a\r\n4         % matlab.unittest.TestCase.\r\n5         \r\n6         methods(Test)\r\n7             function fibonacciOf5(testCase)\r\n8                 % Evaluate the student's function for n = 5\r\n9                 result = fib(5);\r\n10                testCase.verifyEqual(result, 8, 'Fibonacci(5) should be 8');\r\n11            end\r\n12        end\r\n13    end\r\n<\/pre>\n<p>Let's look at Eric's test file that passes the <tt>studentTestFile<\/tt> test, but in which he completely forgot to implement the <tt>F(n) = F(n-1)+F(n-2)<\/tt> recursion step.<\/p>\n<pre class=\"codeinput\">rmpath(<span class=\"string\">'benjamin'<\/span>);\r\naddpath(<span class=\"string\">'eric'<\/span>);\r\ndbtype <span class=\"string\">fib.m<\/span>\r\n<\/pre>\n<pre class=\"codeoutput\">1     function y = fib(n)\r\n2     if (n ~= round(n)) || n &lt; 0\r\n3         error('N is not an integer!');\r\n4     end\r\n5     y = 1;\r\n\r\n<\/pre>\n<p>It should pass the student unit test.<\/p>\n<pre class=\"codeinput\">results = run(studentTest);\r\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\r\ndisp([num2str(percentPassed), <span class=\"string\">'% Passed.'<\/span>]);\r\n<\/pre>\n<pre class=\"codeoutput\">Running studentTest\r\n..\r\nDone studentTest\r\n__________\r\n\r\n100% Passed.\r\n<\/pre>\n<p>It does NOT pass the instructor unit test because it fails one of the test cases.<\/p>\n<pre class=\"codeinput\">results = run(instructorTest)\r\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\r\ndisp([num2str(percentPassed), <span class=\"string\">'% Passed.'<\/span>]);\r\n<\/pre>\n<pre class=\"codeoutput\">Running instructorTest\r\n..\r\n================================================================================\r\nVerification failed in instructorTest\/fibonacciOf5.\r\n\r\n    ----------------\r\n    Test Diagnostic:\r\n    ----------------\r\n    Fibonacci(5) should be 8\r\n\r\n    ---------------------\r\n    Framework Diagnostic:\r\n    ---------------------\r\n    verifyEqual failed.\r\n    --&gt; NumericComparator failed.\r\n        --&gt; The values are not equal using \"isequaln\".\r\n    \r\n    Actual Value:\r\n             1\r\n    Expected Value:\r\n             8\r\n\r\n    ------------------\r\n    Stack Information:\r\n    ------------------\r\n    In C:\\Program Files\\MATLAB\\R2013a\\toolbox\\matlab\\testframework\\+matlab\\+unittest\\+qualifications\\Verifiable.m (Verifiable.verifyEqual) at 411\r\n    In H:\\Documents\\LOREN\\MyJob\\Art of MATLAB\\instructorTest.m (instructorTest.fibonacciOf5) at 10\r\n================================================================================\r\n.\r\nDone instructorTest\r\n__________\r\n\r\nFailure Summary:\r\n\r\n     Name                         Failed  Incomplete  Reason(s)\r\n    ==========================================================================\r\n     instructorTest\/fibonacciOf5    X                 Failed by verification.\r\n    \r\nresults = \r\n  1x3 TestResult array with properties:\r\n\r\n    Name\r\n    Passed\r\n    Failed\r\n    Incomplete\r\n    Duration\r\nTotals:\r\n   2 Passed, 1 Failed, 0 Incomplete.\r\n   0.028906 seconds testing time.\r\n66.6667% Passed.\r\n<\/pre>\n<p>Benjamin, whose code we tested above, wrote a correct solution to the homework problem.<\/p>\n<pre class=\"codeinput\">rmpath(<span class=\"string\">'eric'<\/span>);\r\naddpath(<span class=\"string\">'benjamin'<\/span>);\r\n\r\nresults = run(instructorTest)\r\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\r\ndisp([num2str(percentPassed), <span class=\"string\">'% Passed.'<\/span>]);\r\n\r\nrmpath(<span class=\"string\">'benjamin'<\/span>);\r\n<\/pre>\n<pre class=\"codeoutput\">Running instructorTest\r\n...\r\nDone instructorTest\r\n__________\r\n\r\nresults = \r\n  1x3 TestResult array with properties:\r\n\r\n    Name\r\n    Passed\r\n    Failed\r\n    Incomplete\r\n    Duration\r\nTotals:\r\n   3 Passed, 0 Failed, 0 Incomplete.\r\n   0.015946 seconds testing time.\r\n100% Passed.\r\n<\/pre>\n<h4>Conclusion<a name=\"b8d2ed9f-3356-4308-bc9b-b269f74b93bd\"><\/a><\/h4>\n<p>In this post, we showed you the basics of using the new MATLAB unit testing infrastructure using homework grading as a use case.<\/p>\n<p>We checked that the student's code worked (by returning the correct answer) for one valid value and worked (by throwing an error) for one invalid value. We also showed how you can use this infrastructure to provide an aid\/check for the students that you can also use as part of your grading.<\/p>\n<p>We hope this brief introduction to the unit testing framework has shown you how you can make use of this feature even if you don't consider yourself a software developer. Let us know in the comments for this post how you might use this new functionality. Or, if you've already tried using matlab.unittest, let us know about your experiences <a href=\"https:\/\/blogs.mathworks.com\/loren\/?p=649#respond\">here<\/a>.<\/p>\n<p><script>\/\/ <![CDATA[\nfunction grabCode_ab73aa9d90954425ba64e5af0214ae47() {\n        \/\/ Remember the title so we can use it in the new page\n        title = document.title;\n\n        \/\/ Break up these strings so that their presence\n        \/\/ in the Javascript doesn't mess up the search for\n        \/\/ the MATLAB code.\n        t1='ab73aa9d90954425ba64e5af0214ae47 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\n        t2='##### ' + 'SOURCE END' + ' #####' + ' ab73aa9d90954425ba64e5af0214ae47';\n    \n        b=document.getElementsByTagName('body')[0];\n        i1=b.innerHTML.indexOf(t1)+t1.length;\n        i2=b.innerHTML.indexOf(t2);\n \n        code_string = b.innerHTML.substring(i1, i2);\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\n\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \n        \/\/ in the XML parser.\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\n        \/\/ doesn't go ahead and substitute the less-than character. \n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\n\n        copyright = 'Copyright 2013 The MathWorks, Inc.';\n\n        w = window.open();\n        d = w.document;\n        d.write('\n\n\n\n\n\n<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\n\n\n\n\n\n\n\\n');\n\n        d.title = title + ' (MATLAB code)';\n        d.close();\n    }\n\/\/ ]]><\/script><\/p>\n<p style=\"text-align: right; font-size: xx-small; font-weight: lighter; font-style: italic; color: gray;\"><a><span style=\"font-size: x-small; font-style: italic;\">Get<br \/>\nthe MATLAB code<noscript>(requires JavaScript)<\/noscript><\/span><\/a><\/p>\n<p>Published with MATLAB\u00ae R2013a<\/p>\n<p class=\"footer\">Published with MATLAB\u00ae R2013a<\/p>\n<\/div>\n<p><!--\nab73aa9d90954425ba64e5af0214ae47 ##### SOURCE BEGIN #####\n%% Using the MATLAB Unit Testing Infrastructure for Grading Assignments\n% Steven Lord, Andy Campbell, and David Hruska are members of the Quality\n% Engineering group at MathWorks who are guest blogging today to introduce\n% a new feature in R2013a, the MATLAB unit testing infrastructure. There\n% are several submissions on the MATLAB Central File Exchange related to\n% unit testing of |MATLAB| code. Blogger Steve Eddins wrote one highly\n% rated\n% <https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/22846-matlab-xunit-test-framework % example> back in 2009. In release R2013a, MathWorks included in |MATLAB|\n% itself a |MATLAB| implementation of the industry-standard xUnit testing\n% framework.\n%\n% If you're not a software developer, you may be wondering if this feature\n% will be of any use to you. In this post, we will describe one way someone\n% who may not consider themselves a software developer may be able to take\n% advantage of this framework using the example of a professor grading\n% students' homework submissions. That's not to say that the developers in\n% the audience should move on to the next post; you can use these tools to\n% test your own code just like a professor can use them to test code\n% written by his or her students.\n%\n% There is a great deal of functionality in this feature that we will not\n% show here. For more information we refer you to the\n% <https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html % MATLAB Unit Testing Framework> documentation.\n\n%% Background\n% In order to use this feature, you should be aware of how to define simple\n% |MATLAB| classes in _classdef_ files, how to define a class that inherits\n% from another, and how to specify attributes for methods and properties of\n% those classes. The\n% <https:\/\/www.mathworks.com\/help\/matlab\/object-oriented-programming.html % object-oriented programming> documentation describes these capabilities.\n\n%% Problem Statement\n% As a professor in an introductory programming class, you want your\n% students to write a program to compute Fibonacci numbers. The exact\n% problem statement you give the students is:\n%\n%  Create a function \"fib\" that accepts a nonnegative integer n and returns\n%  the nth Fibonacci number. The Fibonacci numbers are generated by this\n%  relationship:\n%\n%  F(0) = 1\n%  F(1) = 1\n%  F(n) = F(n-1) + F(n-2) for integer n > 1\n%\n%  Your function should throw an error if n is not a nonnegative integer.\n\n%% Basic Unit Test\n% The most basic |MATLAB| unit test is a |MATLAB| _classdef_ class file\n% that inherits from the |matlab.unittest.TestCase| class. Throughout the\n% rest of this post we will add additional pieces to this basic framework to\n% increase the capability of this test and will change its name to reflect\n% its increased functionality.\ndbtype basicTest.m\n\n%%\ntest = basicTest\n\n%% Running a Test\n% To run the test, we can simply pass |test| to the |run| function. There\n% are more advanced ways that make it easier to run a group of tests, but\n% for our purposes (checking one student's answer at a time) this will be\n% sufficient. When you move to checking multiple students' answers at a\n% time, you can use |run| inside a |for| loop.\n%\n% Since |basicTest| doesn't actually validate the output from the student's\n% function, it doesn't take very long to execute.\nresults = run(test)\n\n%%\n% Let's say that a student named Thomas submitted a function |fib.m| as his\n% solution to this assignment. Thomas's code is stored in a sub-folder\n% named |thomas|. To set up our test to check Thomas's answer, we add the\n% folder holding his code to the path.\naddpath('thomas');\ndbtype fib.m\n\n%% Test that F(0) Equals 1\n% The |basicTest| is a valid test class, and we can run it, but it doesn't\n% actually perform any validation of the student's test file. The methods\n% that will perform that validation need to be written in a _methods_ block\n% that has the attribute |Test| specified.\n%\n% The |matlab.unittest.TestCase| class includes qualification methods that\n% you can use to test various qualities of the results returned by the\n% student files. The qualification method that you will likely use most\n% frequently is the |verifyEqual| method, which passes if the two values\n% you pass into it are equal and reports a test failure if they are not.\n%\n% The\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcaseclass.html % documentation> for the |matlab.unittest.TestCase| class lists many other\n% qualification methods that you can use to perform other types of\n% validation, including testing the data type and size of the results;\n% matching a string result to an expected string; testing that a given\n% section of code throws a specific errors or issues a specific warning;\n% and many more.\n%\n% This simple test builds upon |generalTest| by adding a test method that\n% checks that the student's function returns the value 1 when called with\n% the input 0.\ndbtype simpleTest.m\n\n%%\n% Thomas's solution to the assignment satisfies this basic check. We can\n% use the results returned from |run| to display the percentage of the\n% tests that pass.\nresults = run(simpleTest)\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\ndisp([num2str(percentPassed), '% Passed.']);\n\n%% Test that F(pi) Throws an Error\n% Now that we have a basic positive test in place we can add in a test that\n% checks the behavior of the student's function when passed a non-integer\n% value (like |n = pi|) as input. The assignment stated that when called\n% with a non-integer value, the student's function should error. Since the\n% assignment doesn't require a specific error to be thrown, the test passes\n% as long as |fib(pi)| throws any exception.\ndbtype errorCaseTest.m\n\n%%\n% Thomas forgot to include a check for a non-integer valued input in his\n% function, so our test should indicate that by reporting a failure.\nresults = run(errorCaseTest)\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\ndisp([num2str(percentPassed), '% Passed.']);\n\n%%\n% Another student, Benjamin, checked for a non-integer value in his code as\n% you can see on line 2.\nrmpath('thomas');\naddpath('benjamin');\ndbtype fib.m\n\n%%\n% Benjamin's code passed both the test implemented in the\n% |fibonacciOfZeroShouldBeOne| method (which we copied into |errorCaseTest|\n% from |simpleTest|) and the new test case implemented in the\n% |fibonacciOfNonintegerShouldError| method.\nresults = run(errorCaseTest)\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\ndisp([num2str(percentPassed), '% Passed.']);\n\n%% Basic Test for Students, Advanced Tests for Instructor\n% The problem statement given earlier in this post is a plain text\n% description of the homework assignment we assigned to the students. We\n% can also state the problem for the students in code (if they're using\n% release R2013a or later) by giving them a test file they can run just\n% like |simpleTest| or |errorCaseTest|. They can directly use this\n% \"requirement test\" to ensure their functions satisfy the requirements of\n% the assignment.\ndbtype studentTest.m\n\n%%\n% In order for the student's code to pass the assignment, it will need to\n% pass the test cases given in the |studentTest| unit test. However, we\n% don't want to use |studentTest| as the _only_ check of the student's\n% code. If we did, the student could write their function to cover only the\n% test cases in the student test file.\n%\n% We could solve this problem by having two separate test files, one\n% containing the student test cases and one containing additional test\n% cases the instructor uses in the grading process. Can we avoid having to\n% run both test files manually or duplicating the code from the student\n% test cases in the instructor test? Yes!\n%\n% To do so, we write an instructor test file to incorporate, through\n% inheritance, the student test file. We can then add additional test\n% cases to the instructor test file. When we run this test it should run\n% three test cases; two inherited from |studentTest|,\n% |fibonacciOfZeroShouldBeOne| and |fibonacciOfNonintegerShouldError|, and\n% one from |instructorTest| itself, |fibonacciOf5|.\ndbtype instructorTest.m\n\n%%\n% Let's look at Eric's test file that passes the\n% |studentTestFile| test, but in which he completely forgot to implement\n% the |F(n) = F(n-1)+F(n-2)| recursion step.\nrmpath('benjamin');\naddpath('eric');\ndbtype fib.m\n\n%%\n% It should pass the student unit test.\nresults = run(studentTest);\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\ndisp([num2str(percentPassed), '% Passed.']);\n\n%%\n% It does NOT pass the instructor unit test because it fails one of the\n% test cases.\nresults = run(instructorTest)\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\ndisp([num2str(percentPassed), '% Passed.']);\n\n%%\n% Benjamin, whose code we tested above, wrote a correct solution to the\n% homework problem.\nrmpath('eric');\naddpath('benjamin');\n\nresults = run(instructorTest)\npercentPassed = 100 * nnz([results.Passed]) \/ numel(results);\ndisp([num2str(percentPassed), '% Passed.']);\n\nrmpath('benjamin');\n\n%% Conclusion\n% In this post, we showed you the basics of using the new MATLAB unit\n% testing infrastructure using homework grading as a use case.\n%\n% We checked that the student's code worked (by returning the correct\n% answer) for one valid value and worked (by throwing an error) for one\n% invalid value. We also showed how you can use this infrastructure to\n% provide an aid\/check for the students that you can also use as part of\n% your grading.\n%\n% We hope this brief introduction to the unit testing framework has shown\n% you how you can make use of this feature even if you don't consider\n% yourself a software developer. Let us know in the comments for this post\n% how you might use this new functionality. Or, if you've already tried\n% using matlab.unittest, let us know about your experiences\n% <https:\/\/blogs.mathworks.com\/loren\/?p=649#respond here>.\n##### SOURCE END ##### ab73aa9d90954425ba64e5af0214ae47\n--><\/p>\n","protected":false},"excerpt":{"rendered":"<p><!--introduction-->Steven Lord, Andy Campbell, and David Hruska are members of the Quality Engineering group at MathWorks who are guest blogging today to introduce a new feature in R2013a, the MATLAB unit testing infrastructure. There are several submissions on the MATLAB Central File Exchange related to unit testing of <tt>MATLAB<\/tt> code. Blogger Steve Eddins wrote one highly rated example back in 2009. In release R2013a, MathWorks included in <tt>MATLAB<\/tt> itself a <tt>MATLAB<\/tt> implementation of the industry-standard xUnit testing framework.<\/p>\n<p>If you're not a software developer, you may be wondering if this feature will be of any use to you. In this post, we will describe one way someone who may not consider themselves a software developer may be able to take advantage of this framework using the example of a professor grading students' homework submissions. That's not to say that the developers in the audience should move on to the next post; you can use these tools to test your own code just like a professor can use them to test code written by his or her students.<\/p>\n<p>There is a great deal of functionality in this feature that we will not show here. For more information we refer you to the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html\">MATLAB Unit Testing Framework<\/a> documentation.<\/p>\n<p><!--\/introduction-->... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/loren\/2013\/03\/14\/using-the-matlab-unit-testing-infrastructure-for-grading-assignments\/\">read more >><\/a><\/p>\n","protected":false},"author":39,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[39,6],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/649"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/comments?post=649"}],"version-history":[{"count":6,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/649\/revisions"}],"predecessor-version":[{"id":1860,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/649\/revisions\/1860"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/media?parent=649"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/categories?post=649"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/tags?post=649"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}