Goodness time flies! It has been a while since we've had a blog update, While I really would like to keep the conversation we have going on here moving, this is how I would like you to envision what I have been up to recently:

That's right...just think of me as your code monkey, tirelessly pushing forward developer oriented infrastructure (and blog topics). We do indeed have a lot of exciting things we are working on in MATLAB that we will definitely discuss here when the time comes. I can't wait.

However, there are already so many discussions to explore. Today I wanted to highlight another Continuous Integration workflow improvement that is available now that R2015b has been released.

The Beauty is in the Details

If you remember, MATLAB can connect to CI Systems like Jenkins™ which can be configured to run your tests periodically and/or whenever files are modified in your source control system. This is done using the TAPPlugin for the MATLAB TestRunner. This plugin has been enhanced in R2015b to show the diagnostic information in the TAP stream for an improved display in the CI reports. For example, with absolutely no change in the script discussed in the earlier post, the Jenkins output now includes the diagnostics (we heard you Aditya!). So, for the following failing test:

classdef NumericsTest < matlab.unittest.TestCase
    % Copyright 2012 The MathWorks, Inc.
    
    methods(Test)
        function shouldReallyBePi(testCase)
            wannabePi = 22/7;
            testCase.verifyEqual(wannabePi, pi, 'RelTol', 1e-8, ...
                'I am not gonna allow 22/7 as a stand in for pi!');
        end
    end
end

We now see the helpful diagnostics right in the Jenkins reports.

Of course this may differ depending on how your particular CI system presents the TAP output, but MATLAB is certainly providing a richer TAP stream to begin with.

A Horse of a Different Format

Speaking of different CI systems, while Jenkins is great to highlight because it is simple to show in a short blog post and is popular to boot, but there are of course many other CI systems out there. One of the reasons I am a supporter of the TAP format is because it aims to be language independent. However, unfortunately there are some CI systems that simply don't have the capability to process TAP streams. The de-facto JUnit XML format (which is in fact the xml output produced by the Apache Ant JUnit task) is very widespread and supported by some CI systems that do not support the TAP protocol. Once again, R2015b to the rescue because test results can now be exported to the JUnit format using the new XMLPlugin. This allows a few benefits:

Before that though, it would be great to have the same detailed diagnostics as the new version of the TAPPlugin and it does! To see this you can just install the MATLAB XMLPlugin onto the TestRunner as follows:

import matlab.unittest.TestSuite;
import matlab.unittest.TestRunner;
import matlab.unittest.plugins.TAPPlugin;
import matlab.unittest.plugins.ToFile;

import matlab.unittest.plugins.XMLPlugin;

try
    suite = TestSuite.fromPackage('testcases','IncludingSubpackages',true);
    
    runner = TestRunner.withTextOutput();
    
    % Add the JUnit XML plugin
    xmlFile = fullfile(getenv('WORKSPACE'), 'testResults.xml');
    runner.addPlugin(XMLPlugin.producingJUnitFormat(xmlFile));
    
    % Also add the TAP plugin because why not? 
    tapFile = fullfile(getenv('WORKSPACE'), 'testResults.tap');
    runner.addPlugin(TAPPlugin.producingOriginalFormat(ToFile(tapFile)));
        
    results = runner.run(suite);
    display(results);
catch e
    disp(getReport(e,'extended'));
    exit(1);
end
exit;

Then configure a post-build step in Jenkins to publish the "JUnit" results:

...and viola, we can see the diagnostics for the JUnit publishing step as well:

So which output format is the favorite child? You tell me! Did you notice the improved diagnostics of the TAPPlugin? Can you benefit from the JUnit-style XML format? If so I'd love to hear some details!

Get the MATLAB code

Published with MATLAB® R2015b