Today it is my pleasure to introduce my MathWorks colleague Anoush Najarian. Anoush is an engineering manager on the MATLAB performance team and she has been instrumental in many great projects such as the performance testing framework we have been blogging about as well as the significant performance improvements with the new MATLAB execution engine in R2015b. As it turns out she is also voraciously involved in the maker community and she can be seen doing all sorts of fun Internet of Things projects like A Tweeting Plant. That's good news for us because she can now use her performance know-how along with her familiarity with the ThingSpeak IoT platform to demonstrate how easy it is to use the platform to track the performance trends of your code over time. So without further ado let's hear from Anoush:

It's important to visualize results from our performance test runs! Why? People are awesome at spotting patterns, and learn better from pictures.

I love using ThingSpeak for maker projects. For this post, I thought it would be fun and useful to get ThingSpeak to help track and analyze trends from MATLAB Performance Testing Framework runs. We're going to borrow an example Andy discussed in Peer Comparison.

Let's figure out how to track the performance of Sarah's code over time.

First, let's run the Initial Code:

measResult = runperf('comparisonTest/InitialCode');
samples = measResult.Samples

Running comparisonTest
........
Done comparisonTest
__________


samples = 

               Name               MeasuredTime         Timestamp                    Host               Platform           Version                      RunIdentifier            
    __________________________    ____________    ____________________    _________________________    ________    _____________________    ____________________________________

    comparisonTest/InitialCode    23.443          29-Jun-2016 16:44:18    Anoushs-MacBook-Pro.local    maci64      9.0.0.341360 (R2016a)    f0918bed-906d-4fed-93c5-dd9bb2241836
    comparisonTest/InitialCode    23.662          29-Jun-2016 16:44:41    Anoushs-MacBook-Pro.local    maci64      9.0.0.341360 (R2016a)    f0918bed-906d-4fed-93c5-dd9bb2241836
    comparisonTest/InitialCode    24.057          29-Jun-2016 16:45:05    Anoushs-MacBook-Pro.local    maci64      9.0.0.341360 (R2016a)    f0918bed-906d-4fed-93c5-dd9bb2241836
    comparisonTest/InitialCode    24.207          29-Jun-2016 16:45:30    Anoushs-MacBook-Pro.local    maci64      9.0.0.341360 (R2016a)    f0918bed-906d-4fed-93c5-dd9bb2241836

We set up our ThingSpeak channel fields to mimic the data structure of the performance testing framework. (Take a look at Getting Started with ThingSpeak!) You can use our public channel to post your data, all the code in this post will work.

In order to post our data to ThingSpeak, we need to format it a bit, and provide a Write Key for our channel:

dataTable = table(samples.Timestamp, cellstr(samples.Platform), cellstr(samples.Host), cellstr(samples.Version), cellstr(samples.Name), cellstr(samples.RunIdentifier), samples.MeasuredTime);
channelID = 82845;
writeKey = 'H952GH69VMDB4C2X';
thingSpeakWrite(channelID, dataTable, 'WriteKey', writeKey);

We can include this script in our continuous integration workflow, or run it manually whenever we make changes to our algorithm. The data is uploaded to ThingSpeak, and persists there. We can manipulate it then using a collection of cool MATLAB tools on ThingSpeak.

We gradually implemented all of Sarah's optimizations. Here is what our trend looks like so far this year; we included a bonus trend for the past couple of months:

Here's the code running on ThingSpeak, driving the first of our MATLAB Visualizations. You also can run it in your MATLAB session, and get a plot locally!

readChannelID = 82845;
data = thingSpeakRead(readChannelID, 'Fields', [4, 6], 'NumPoints', 649, 'OutputFormat', 'table');
thingSpeakPlot(data.Timestamps, cellfun(@str2num, data.MeasuredTime), 'XLabel', 'Date', 'YLabel', ...
    'Measured Time, seconds', 'Grid', 'on', 'Marker', 'o', 'LineStyle','-','LineWidth',2, ...
    'Title', 'MATLAB Performance Testing Framework Trend For 2016');

If we hover over the data points in the trend plots, or zoom in, we will spot the key optimizations that helped the performance of Sarah's code. Initial Code measured at around 23.8s:

On February 20, we switched to Code with Preallocation, speeding up to around 22.3s, or by 6%:

On May 2, we implemented the optimization to Vectorize the Inner Two loops; this sped up the code 100+-fold, to 0.2s.

Finally, on June 9, we implemented Vectorize the Inner Three Loops, and the code sped up to 0.06s, or by 67%.

In all, our trends capture pretty awesome 400+-fold performance improvement in the past few months!

What's next? We can use MATLAB Analysis and Visualization tools on ThingSpeak for more advanced data analysis, like looking at error bars and normalizing performance across a suite of tests. We can set up email alerts in response to changes in our performance trends.

Let us know in comments how you've been tracking performance over time, and what visualizations and alerts would be useful to you!

Get the MATLAB code

Published with MATLAB® R2016a