{"id":6441,"date":"2017-05-17T10:04:52","date_gmt":"2017-05-17T15:04:52","guid":{"rendered":"https:\/\/blogs.mathworks.com\/simulink\/?p=6441"},"modified":"2017-05-18T15:01:33","modified_gmt":"2017-05-18T20:01:33","slug":"simulink-and-the-matlab-unit-testing-framework","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/simulink\/2017\/05\/17\/simulink-and-the-matlab-unit-testing-framework\/","title":{"rendered":"Simulink and the MATLAB Unit Testing Framework"},"content":{"rendered":"<p><!--introduction--><br \/>\nToday, I am happy to welcome guest blogger Ajay Puvvala to talk about testing.<\/p>\n<p>Some time ago, Ajay passed by my desk and offered to blog about the <a href=\"https:\/\/www.mathworks.com\/help\/sltest\/ug\/run-test-files-using-matlab-unit-test.html\">tight integratio<\/a>n between <a href=\"https:\/\/www.mathworks.com\/products\/simulink-test.html\">Simulink Test<\/a> and <a href=\"https:\/\/www.mathworks.com\/videos\/matlab-unit-testing-framework-74975.html\">MATLAB Unit Testing Framework<\/a>, which got introduced in R2016b.<\/p>\n<p>My answer was: Yes, of course... but before, we need to introduce what the MATLAB Unit Testing Framework is, and how it can be used in a Simulink context.<\/p>\n<p>This is what Ajay will be describing today.<\/p>\n<p><!--\/introduction--><\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/apuvvala.jpg\" alt=\"Guest Blogger Ajay Puvvala\" \/><\/p>\n<p><strong>The Goal<\/strong><\/p>\n<p>As an example, we will test if the output of the code generated for a simple model matches the results of a normal mode simulation. For that, we create a harness model, and using model referencing we simulate our simple model both in normal and in Software-in-the-Loop mode.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/challenge.png\" alt=\"Model to be tested\" \/><\/p>\n<p><strong>Script, Function or Class based<\/strong><\/p>\n<p>Depending on your preferences and needs, MATLAB Unit Testing offers <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html\">three frameworks<\/a>: <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/script-based-unit-tests.html\">Script-Based<\/a>, <a title=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/write-function-based-unit-tests-.html (link no longer works)\">Function-Based<\/a> and <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/class-based-unit-tests.html\">Class-Based<\/a>. The flexibility and set of features of those frameworks increase in that order.<\/p>\n<p>For this post, we decided to go for function-based, since it offers everything we need, with the convenience of procedural programming.<\/p>\n<p>Let's begin by the simplest possible test one could write:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/simplestTest.png\" alt=\"Simple test\" \/><\/p>\n<p>What is in this test?<\/p>\n<ul>\n<li>The main function in our test uses <tt><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/functiontests.html\">functiontests<\/a><\/tt> to return an array of tests, which are the other local functions in this file taking a test handle as input.<\/li>\n<li>The function <tt>testEquivalence<\/tt> is our main test, it simulates the model in normal and SIL mode, and compares the results.<\/li>\n<li>We use <tt><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.testcase.addteardown.html\">addTeardown<\/a><\/tt> to do the cleanup once the test is completed. In this case, this means closing the model.<\/li>\n<li>We use <tt><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.qualifications.verifiable.verifyequal.html\">verifyEqual<\/a><\/tt> from the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab_prog\/types-of-qualifications.html\">MATLAB Unit Test qualification library<\/a> to compare the results.<\/li>\n<\/ul>\n<p>When ready, use the <tt><a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/runtests.html\">runtests<\/a><\/tt> function to run all the tests in <tt>exampleTest.m<\/tt>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/firstTestResult.png\" alt=\"First test\" \/><\/p>\n<p>As you can see, our test failed. To make this story more interesting, I purposely configured the generated code to link against a math library where the implementation of <tt>sin<\/tt> is slightly different than the one used by Simulink in normal mode. In case you were not aware, there are various math libraries implementing trigonometric functions in different ways, giving slightly different results.<\/p>\n<p><strong>Diagnostics<\/strong><\/p>\n<p>The failure diagnostics show a table with numerical mismatches. But as they say, an image is worth a thousand words. So, how can we have better diagnostics? Luckily, qualification methods <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.qualifications.verifiable.verifyequal.html#inputarg_diagnostic\">accept a user diagnostics for additional diagnosis<\/a>. It could be a string or character array, function handle or <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.unittest.diagnostics.diagnostic-class.html\">matlab.unittest.diagnostics.Diagnostic<\/a> object. Let\u2019s create a diagnostic that can help us visualize these signal mismatches rather than looking at a bunch of numbers.<\/p>\n<p>We add the following function to our test file <tt>exampleTest.m<\/tt> to launch SDI and display a comparison in case the test fails.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/compareRuns.png\" alt=\"Adding a Diagnostic function camparing runs\" \/><\/p>\n<p>and we specify this function as additional argument to verifyEqual<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/addDiagnostics.png\" alt=\"Diagnostic function camparing runs\" \/><\/p>\n<p>This time, when the test finishes, SDI shows us how different the signals are:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/sSDI_results.png\" alt=\"Result of the Diagnostic function camparing runs\" \/><\/p>\n<p><strong>Tolerances<\/strong><\/p>\n<p>As we all know, doing bit-to-bit comparison with floating-point signals is usually a bad idea. You typically end up being to sensitive to floating-point roundoff errors. Instead, you want to verify that the results are equivalent within a certain tolerance.<\/p>\n<p>For that, we can pass additional arguments to <tt>verifyEqual<\/tt> to specify relative and absolute tolerances<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/absTol.png\" alt=\"Adding absolute tolerance\" \/><\/p>\n<p>This time, the test passes!<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/passedTest.png\" alt=\"Passed Test\" \/><\/p>\n<p><strong>What's next?<\/strong><\/p>\n<p>As I mentioned previously, if you want to read more about testing in general, I recommend that you visit the <a href=\"https:\/\/blogs.mathworks.com\/developer\/\">Developer Zone blog<\/a>, which is dedicated to this topic.<\/p>\n<p>I also recommend that you go through the <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/matlab-unit-test-framework.html\">Testing Frameworks documentation<\/a> to see all the different features available to customize and enhance the example we went through in this blog post.<\/p>\n<p>Next week, we will look at how using Simulink Test\u2019s Test Manager can simplify the testing of Simulink models. Stay tuned!<\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/images\/simulink\/2017Q2\/passedTest.png\" onError=\"this.style.display ='none';\" \/><\/div>\n<p><!--introduction--><br \/>\nToday, I am happy to welcome guest blogger Ajay Puvvala to talk about testing.<\/p>\n<p>Some time ago, Ajay passed by my desk and offered to blog about the <a href=\"https:\/\/www.mathworks.com\/help\/sltest\/ug\/run-test-files-using-matlab-unit-test.html\">tight integratio<\/a>n between <a href=\"https:\/\/www.mathworks.com\/products\/simulink-test.html\">Simulink Test<\/a> and <a href=\"https:\/\/www.mathworks.com\/videos\/matlab-unit-testing-framework-74975.html\">MATLAB Unit Testing Framework<\/a>, which got introduced in R2016b.<\/p>\n<p>My answer was: Yes, of course... but before, we need to introduce what the MATLAB Unit Testing Framework is, and how it can be used in a Simulink context.<\/p>\n<p>This is what Ajay will be describing today.<\/p>\n<p><!--\/introduction-->... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/simulink\/2017\/05\/17\/simulink-and-the-matlab-unit-testing-framework\/\">read more >><\/a><\/p>\n","protected":false},"author":41,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[121,21,24],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts\/6441"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/users\/41"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/comments?post=6441"}],"version-history":[{"count":36,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts\/6441\/revisions"}],"predecessor-version":[{"id":6583,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/posts\/6441\/revisions\/6583"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/media?parent=6441"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/categories?post=6441"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/simulink\/wp-json\/wp\/v2\/tags?post=6441"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}