Here is Stephen Carter once again to describe an exciting feature in R2016b for test result analysis, reporting, and archiving. Take it away Stephen!

In my previous post, I demonstrated that you can consume our secret sauce programmatically via the new Details property on your TestResult output starting in R2016a. Like I mentioned in that post, this new way of accessing diagnostics will come in handy when you accidentally clear your diagnostic output from the screen by typing clc. But what if you are one of those users who, in addition to typing clc, also have a habit of typing clear all?

If you type clear all, or if you accidentally wipe out your TestResult array because it was stored in the ans variable, or if you end your MATLAB session, or whatever the case may be... if you lose your test results the only way to get them back is by rerunning your tests. So how can you avoid this situation?

I suggest writing your diagnostic output to a file instead of the command window. Now many of you might already be doing this, especially if you have been integrating your diagnostics with a continuous integration system as discussed in recent posts. But for those of you not integrating with a CI system, or who may have forgotten how to save your output to a file, it is as simple as using the ToFile stream when constructing your plugins which produce output. For example:

    import matlab.unittest.plugins.*;    
    import matlab.unittest.TestRunner;  
    
    runner = TestRunner.withTextOutput;
    fileStream = ToFile('myTestRunResults.txt');
    runner.addPlugin(LoggingPlugin.withVerbosity(1,fileStream));
    runner.addPlugin(TestRunProgressPlugin.withVerbosity(2,fileStream));
    runner.addPlugin(FailureDiagnosticsPlugin(fileStream));
    % ...

But this isn't anything new. So why am I talking about it? Well, first I wanted to point out that getting in the habit of saving your results to a file will help you avoid losing the diagnostic results of your test run. But I must ask you... do you like reading plain text? Wouldn't you rather be able to publish your test results to a nice looking report?

As another motivating scenario, imagine you would like to share your test results with another person. Although you always have the option to let people look at your computer screen or look at your saved text file, it isn't very professional. Another option would be to show them your results in your CI system, but what if you haven't submitted your results to your CI system yet or what if they don't have access to your CI system? Or in an extreme case, what if you find yourself wanting to share your results 20 years later. There is a good chance your CI system would no longer exist anymore. We need a different way to serve the diagnostics to address these use cases.

This is where a professionally formatted DOCX or PDF document would come to the rescue. Well you are in luck! Recently, we've begun putting efforts into report generation and starting in R2016b we now offer a TestReportPlugin. So if you would like to either archive your test results, more easily navigate your test results, or impress your boss with a professional looking document which demonstrates all the great testing you did, you can do the following:

    import matlab.unittest.TestRunner;
    import matlab.unittest.plugins.TestReportPlugin;
    import matlab.unittest.TestSuite;
    
    runner = TestRunner.withTextOutput; %or choose withNoPlugins
    
    reportFile = [tempname '.docx']; %or choose your own location
    plugin = TestReportPlugin.producingDOCX(... %or choose producingPDF
        reportFile,...
        'IncludingPassingDiagnostics',true,...
        'PageOrientation','portrait',... %or choose 'landscape'
        'IncludingCommandWindowText', false);
    runner.addPlugin(plugin);
    
    suite = TestSuite.fromFolder(pwd); %or choose whatever suite you want
    results = runner.run(suite);

This will generate a report file with the details of your test run results. The first page of the document is a cover page to give you a quick look at the overall result of your test run.

You will be able to navigate the document easily using the navigation pane.

The summary and overview sections allow you to easily see what areas of your test suite passed, failed, or were filtered.

These sections also provide you with icons and text in which you can click to easily jump to the details of an individual test.

And of course there is more that I could show you, so go try it out for yourself! We will continue to improve our reporting capabilities in releases to come.

Additionally, we are working hard on providing even more additional ways to serve up our secret sauce to you. So we ask you... what are your use cases and preferences when it comes to diagnostics? Have we added enough salt and pepper? Let us know.

Get the MATLAB code

Published with MATLAB® R2016b