{"id":3044,"date":"2023-08-16T10:51:55","date_gmt":"2023-08-16T14:51:55","guid":{"rendered":"https:\/\/blogs.mathworks.com\/developer\/?p=3044"},"modified":"2023-09-26T06:32:28","modified_gmt":"2023-09-26T10:32:28","slug":"introducing-matlab-test","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/developer\/2023\/08\/16\/introducing-matlab-test\/","title":{"rendered":"Introducing MATLAB Test!"},"content":{"rendered":"
MATLAB R2023a was released back in March, and with it was a new product <\/span>MATLAB Test<\/span><\/a>. In a series of blog posts starting today, I\u2019m going to show you how MATLAB Test can help you improve the quality of your code whether you\u2019re developing a MATLAB Toolbox to share with your colleagues or writing code that will be deployed into a safety critical environment such as for medical devices.<\/span><\/div>

Overview<\/span><\/h2>
There are five key features in the first release of MATLAB Test which I\u2019m going to cover. They will be extended and added to over the coming releases:<\/span><\/div>
  1. Test Manager \u2013 manage your tests and results (today\u2019s topic).<\/span><\/li>
  2. Extended coverage \u2013 measure condition, decision, and modified condition and decision coverage (MCDC) to help ensure your code is fully tested and meets regulatory requirements.<\/span><\/li>
  3. Code quality dashboard \u2013 see an overview of your project\u2019s quality status such as coverage, Code Analyzer warnings, and requirements links.<\/span><\/li>
  4. Equivalence testing with MATLAB Coder and MATLAB Compiler SDK \u2013 easily compare the behaviour of generated C or C++ code, Java or Python packages, or .NET assemblies to that of desktop MATLAB.<\/span><\/li>
  5. Dependency based test selection \u2013 execute only the tests in your suite that are impacting by changes in your code base.<\/span><\/li><\/ol>
    In addition, tool qualification artifacts are available for MATLAB Test in our <\/span>IEC Certification Kit<\/span><\/a> making MATLAB Test suitable for use in the development of high integrity software.<\/span><\/div>

    What\u2019s not changing?<\/span><\/h2>
    Don\u2019t worry! The MATLAB <\/span>unit testing<\/span><\/a> (including <\/span>measuring statement coverage<\/span><\/a>), <\/span>mocking<\/span><\/a>, <\/span>app testing<\/span><\/a>, and <\/span>performance testing<\/span><\/a> frameworks are not changing and remain in base MATLAB. <\/span><\/div>
    In addition, R2023a sees the introduction to base MATLAB of the <\/span>Test Browser<\/span><\/a>, a simple interface to run tests and browse results.<\/span><\/div>
    \"testBrowser.png\"<\/img><\/div>
    Test Browser<\/span><\/div>
    The Test Browser will launch automatically whenever you run tests from a test file using the \u201cRun Tests\u201d (F5) or \u201cRun Current Test\u201d (Ctrl + Enter) buttons. If you run your tests from the command line (<\/span>runtests(\u201ctDemoSmokeTest\u201d)<\/span>), the Test Browser will not launch and you will instead see the classic Command Window output.<\/span><\/div>

    Test Manager<\/span><\/h2>
    The Test Manager allows you to create and manage test suites, view and search results, view requirements links, manage coverage settings, and persist results between MATLAB sessions.<\/span><\/div>
    So what\u2019s the difference between the Test Browser and the Test Manager? <\/span>The<\/span><\/span><\/a> <\/span>Test Browser is a simple viewer for iteratively developing code and tests within a MATLAB session. The Test Manager allows you to manage tests at scale, link to requirements, and persist results between sessions.<\/span><\/div>
    To use Test Manager, you will need to be using a <\/span>Project<\/span><\/a>. (For this blog post, I\u2019m using the shipping <\/span>shortest path example<\/span><\/a>.) The Test Manager needs Projects for dependency information and for persisting test results. You can launch the Test Manager from the Project Toolstrip under the Apps section:<\/span><\/div>
    \"testManagerShortcut.png\"<\/img><\/div>
    Launch the Test Manager from the Project toolstrip.<\/span><\/div>
    When you first launch the Test Manager, it will automatically find all tests in your project. You can add additional columns to the table view (click the green plus icon to the top right of the table) to view test tags and linked requirements (requires the <\/span>Requirements Toolbox<\/span><\/a>).<\/span><\/div>
    \"testManagerWithRequirements.png\"<\/img><\/div>
    Test Manager with \u201cTags\u201d and \u201cRequirements\u201d columns added.<\/span><\/div>
    Once you have run the tests (hit the green play button!), you can view your tests. You can filter the results by their pass\/fail\/incomplete\/not run status, or search based on some text. Here I\u2019ve searched for \u201cinput\u201d which finds all the tests that have the tag \u201cInputTests\u201d.<\/span><\/div>
    \"testManagerSearch.png\"<\/img><\/div>
    Test results filtered to show those that include \u201cinput\u201d in their metadata.<\/span><\/div>

    Collecting coverage information<\/span><\/h2>
    Coverage can be configured using the option in the menu bar:<\/span><\/div>
    \"coverageSettings.png\"<\/img><\/div>
    Coverage settings.<\/span><\/div>
    Whilst the definition of statement coverage is fairly obvious, the differences between the others can be subtle. We\u2019ve added a page in the documentation to explain the <\/span>meaning of each type of coverage<\/span><\/a> as they apply to MATLAB source code. I will also explore it more in a future blog post.<\/span><\/div>
    The coverage recording level in MATLAB Test is cumulative, i.e. MC\/DC also includes Statement, Decision, and Condition. <\/span><\/div>
    Coverage results can be viewed by clicking the adjacent menu item after all tests have been run:<\/span><\/div>
    \"coverageReportButton.png\"<\/img><\/div>
    Access the coverage report.<\/span><\/div>
    (As of R2023a, coverage information is only available when all tests in the current project are run; coverage is not recorded when running a custom test suite. This restriction will be removed in a future release.)<\/span><\/div>
    Coverage is recorded for all files in your project that have the classification of \u201cDesign\u201d. By default, all files in a project will be classified as \u201cDesign\u201d except those that use the unit testing framework which are classified as \u201cTest\u201d. If you want to remove a file from the coverage report, change the classification to something else.<\/span><\/div>
    \"projectLabels.png\"<\/img><\/div>
    Change a file\u2019s classification to something other than \u201cDesign\u201d to exclude it from coverage recording.<\/span><\/div>

    Creating custom test suites<\/span><\/h2>
    The Test Manager provides a graphical interface for creating custom test suites by using this menu option:<\/span><\/div>
    \"createTestSuite.png\"<\/img><\/div>
    Create a custom test suite.<\/span><\/div>
    Here, I\u2019ve created a new test suite to select all tests that have the tag \u201cInputTests\u201d:<\/span><\/div>
    \"testSuiteManager.png\"<\/img><\/div>
    Creating a new test suite for all tests in the project with the tag \u201cInputTests\u201d.<\/span><\/div>
    <\/span><\/div>
    I can then select my suite from the dropdown menu and just run that subset:<\/span><\/div>
    \"testManagerTestSuite.png\"<\/img><\/div>
    Viewing tests that are part of the custom suite.<\/span><\/div>

    Session persistence<\/span><\/h2>
    Finally, since the Test Manager persists the results of tests from session to session, I can close MATLAB, open it up again later, reopen the Test Manager, and the results of my last test run will still be there.<\/span><\/div>

    Summary<\/span><\/h2>
    MATLAB Test\u2019s Test Manager helps you to manage your test suite, coverage, and requirements as you scale up your testing activities.<\/span><\/div>
    In the next post, I\u2019ll take a look at the new coverage options in MATLAB Test.<\/span><\/div>\r\n<\/div>