Loren on the Art of MATLAB

Turn ideas into MATLAB

Run Code Faster With the New MATLAB Execution Engine 17

Posted by Loren Shure,

Today I'd like to introduce you to a guest blogger, Dave Bergstein, who is a MATLAB Product Manager here at MathWorks. Today, Dave will be discussing the new MATLAB execution engine.

Contents

Redesigned Execution

In R2015b we changed the way all MATLAB code is executed with the introduction of the new MATLAB execution engine. The improved architecture runs MATLAB programs faster and provides a platform for future enhancements. And these benefits come without requiring changes to your code.

See what some customers are saying about the new execution engine:

"I work with a wide variety of clients and spend time helping many of them speed up their existing MATLAB code. I was very excited then to see how much the new execution engine helps performance. One client reported that a key performance-critical process now runs twice as fast in R2015b, providing a significant positive impact on their daily operations." Yair Altman
MATLAB expert and author of the book Accelerating MATLAB Performanc (CRC Press)

"The new MATLAB execution engine in R2015 halved the execution time of our MATLAB code." Robert Danforth
Director -“ Engineering, Analytical Modeling & Test Labs, Kohler Co.

"My apps -“ 1D and 2D finite volume for heat transfer and fluid flow -“ perform significantly better -“ about 35% better. Good job!" Enrico Nobile
Professor of Thermodynamics and Heat Transfer, Dept. of Engineering and Architecture, University of Trieste, ITALY

What It Means For You: Better Performance, Quality, and Extensibility

The new execution engine was designed with performance in mind from the beginning. The following are some specific language areas with the most significant improvements:

  • Function Calls: Function call overhead is greatly reduced; there is no longer an appreciable performance penalty for organizing code into many small functions.
  • Object-Oriented Features: Many object-oriented operations execute faster.
  • Element-Wise Math Operations: Many element-wise math operations execute faster.

A number of factors account for the speedup in these areas and others, including improved just-in-time (JIT) compilation of MATLAB code. Redesigning the execution engine was also a time to revisit and resolve old language imperfections and bugs, improving quality. And it was an opportunity to design the engine for extensibility, facilitating future enhancements and new features to come.

What is JIT Compilation?

The new MATLAB execution engine can JIT compile all MATLAB code and this is an important factor in how it executes code faster. Programming languages like C++ are so-called compiled languages. C++ code goes through a compile step initiated by the author. This compile step generates machine code which executes directly on the target hardware. And compilers can often optimize the generated code to help applications execute faster.

Interpreted languages are executed command-by-command by an interpreter program and don't require an explicit compile step. For engineers and scientists working interactively with their data, it can be very helpful to execute commands this way, altering the sequence of commands as they work. Although this style of execution can be very effective for exploring ideas, it can be slow when executing blocks of code.

MATLAB provides the best of both worlds by compiling MATLAB code on-the-fly, or just-in-time. MATLAB code is compiled whether it be in classes, functions, scripts, or simply at the command line. There is no explicit compile step for the user to initiate and MATLAB code can be executed in blocks as small as one line at a time. The MATLAB JIT compiler generates native machine level code that is optimized for the MATLAB code being executed and for the specific hardware platform.

MATLAB has employed JIT compilation since release R13 in 2002. The original JIT compiler was designed as a supplement to the MATLAB interpreter, and supported a limited subset of the language. The new execution engine was designed with a focus on JIT compilation, where JIT compilation is an integral part of the engine and able to compile the entire MATLAB language.

Example Improvement

This example gives a sense of the benefit of the new JIT compiler. It demonstrates a difference between the previous JIT compiler and the new JIT compiler which can compile all MATLAB code.

The function test1 calls the function foo1 in a for-loop which performs a straightforward computation using the looping variable ii. The function foo1 first checks its input value and can issue an error using the error command if the input is out of range. The error confidition won't occur in this example, but such input checking is a common practice that helps when functions get reused.

function Test1
x = zeros(1,1e6);
for ii = 1:1e6
    x(ii) = foo1(ii);
end
end

function y = foo1(x)
if (x<0)
    error('x must be positive');
end
y = 2*x.^2+3*x+4;
end

I recorded the following times for Test1 using the timeit command on my laptop [1]:

  • Test1 with R2015a: 1.7887 seconds
  • Test1 with R2015b: 0.0435 seconds

The new execution engine is more than 40 times faster executing this example!

Next, consider what happens when we make a slight change to the test. Test2 is the same as Test1 except that it calls foo2 which issues the possible error through another function.

function Test2
x = zeros(1,1e6);
for ii = 1:1e6
    x(ii) = foo2(ii);
end
end

function y = foo2(x)
if (x<0)
    displayError;
end
y = 2*x.^2+3*x+4;
end

function displayError
error('x must be positive');
end

I recorded the following times for Test2 using the timeit command:

  • Test2 with R2015a: 3.8667 seconds
  • Test2 with R2015b: 0.0430 seconds

With R2015a Test2 takes twice as long to run as Test1 did (3.8667 seconds vs 1.7887 seconds). The reason for the slowdown is that the added code complexity limits the ability of the previous JIT compiler to compile the code. With R2015b the execution time between Test1 and Test2 is nearly the same, demonstrating the ability of the new execution engine to JIT compile all MATLAB code. The result is that Test2 executes nearly 90 times faster with the new execution engine in R2015b.

Improving Performance

MATLAB spends some amount of time compiling the code. The time to compile is typically small relative to the time to execute the code. And when code is run repeatedly in a MATLAB session without being cleared, you get the greatest benefit since the code only needs to be compiled once. Examples of running code repeatedly are for-loops or rerunning unmodified MATLAB files. Knowing this, consider the following ways to improve the performance of your code:

  1. Avoid using clear all since it clears the compiled code. Use clear instead when you want to just clear your data and avoid re-incurring the time MATLAB spends compiling.
  2. Modularize your code into multiple files so that when you make edits, it reduces the amount of code that gets recompiled on the next run. Modularizing code into separate functions and classes can also help improve code readability, reusability, and maintainability.

Results

We collected and tested 76 performance-sensitive user applications to evaluate the performance of the new engine. The programs tested range in size from a few dozen lines of code to hundreds of lines of code and cover applications from device simulation to finance. The histogram below shows the number of applications (or tests) that ran some amount faster or slower in MATLAB 15b compared to 15a. From the histogram you can see that there is strong shift to the right, indicating faster performance. These applications ran on average 40% faster with the new execution engine. And over a dozen of the applications ran at least twice as fast.

How Fast is Your Code?

Test your code with the redesigned MATLAB execution engine by downloading and installing R2015b. You'll be taking advantage of the MATLAB execution engine as soon as you start running your code. For tips on improving code performance, see Techniques for Improving Performance . To learn more about assessing the performance of your code, see Measure Performance of Your Program . And share with us how your code executes with the new execution engine by leaving a note here!

[1] For a great discussion of timing code, see Loren's Dec blog post .


Get the MATLAB code

Published with MATLAB® R2015b

Note

Comments are closed.

17 CommentsOldest to Newest

Alexandre Kozlov replied on : 1 of 17
"Object-Oriented Features: Many object-oriented operations execute faster." - can you explain it with more details, please?
Royi replied on : 2 of 17
Performance increase is great. But we need more. Julia set the standard with C / C++ performance, we want it in MATLAB as well.
Philip replied on : 3 of 17
I've previously posted this https://www.mathworks.com/matlabcentral/answers/239817-matlab-s-r2015b-new-jit-experiences-a-severe-degradation-in-speed-in-the-following-example-but-the, but the new JIT has simple cases where it's much slower than the old JIT. In the example provided, the slowdown is over 3 times. It feels like the new JIT is faster in some areas and slower in some areas; which makes it a gamble as to if it'll actually improve the speed of your code.
Brad Stiritz replied on : 4 of 17
Philip, Thanks for linking to your code example, very interesting. I sympathize with your surprise and disappointment. Unfortunately, there will always be cases where an optimizing compiler will make poor choices. I believe this is ultimately due to the so-called "halting problem" [1]. In simple terms, the compiler has to set a limit for itself on how much time to spend analyzing the input code, because in theory it could spend an indefinite amount of time. I saw a similar example [2] involving Intel's "Composer" C++ compiler. I don't know all the competitors, but I would think Composer likely sets the gold standard on Intel-based hardware. In any case, my feeling is that if Composer can suffer bad edge cases, then any compiler can. I also think that over time, TMW will study anomalies like yours very carefully, and at least explore possibilities to improve the engine accordingly. References : [1] https://en.wikipedia.org/wiki/Halting_problem [2] http://programmers.stackexchange.com/a/151383/149504
Michal replied on : 5 of 17
The real situation is a bit complicated!!! This optimistic PR-like craps are far away from reality. In this post is clearly shown, that new JIT completely deform profiler results, so finally is not clear, which code(serial and or vectorized) is really more optimized. I think the new JIT engine is definitely not matured product, because recent benchmark results are really confusing. And finally, TMW has to be pay more attention to MATLAB kernel functionality (speed optimization) to catch Julia excellent performance!!!
Dave Bergstein replied on : 6 of 17
@Alexandre: Examples of faster OOP operations are method call overhead (helpful when calling small methods many times) and indexing within or into an object. The following may give a sense of where OO speeds ups can happen. I measure this at 1.7 seconds in R2015a and 0.1 seconds in R2015b.
myclass.m:

classdef myclass
    properties
        Prop1 = 20;
    end
    methods
        function out = method1(obj)
            out = obj.Prop1 * 10;
        end
    end
end

perftest.m:
m = myclass;
for i = 1:10^5
    method1(m);
end
@Royi: Thanks for this input. Some other users have told us about cases where MATLAB is faster than Julia. The new MATLAB execution engine puts us in a great position to continue improving our performance overall and I'm excited by the investments we're making here. (and btw, if you haven't already, optimizing your MATLAB code can sometimes help a surprising amount.) @Philip: Customer feedback since the release seems consistent with the histogram above – a majority of applications run significantly faster. It’s also true that not all code runs faster in the new execution engine. In the old JIT, some simple constructs were highly optimized for performance. Some of these perform similarly in the new JIT, or in cases, a bit slower. The example you shared in your MATLAB Answers post hits a sweet spot for the old JIT. In your example, if you were to either put the code in a function, or vectorize it, it would not only run faster overall, but faster in R2015b than R2015a. That said, I hear your point that this isn’t your real case, just a mockup. This raises a separate issue, which is that small code mockups often don’t represent actual application performance. Perhaps we can follow up offline to learn more details about this case.
Brad Stiritz replied on : 7 of 17
Dave, thanks for your interesting post and for your responses to questioners in the comments. I was disappointed to see the lack of proper indenting for your code example. What is the status on TMW blogs supporting code markup in comments?
Loren Shure replied on : 8 of 17
@Brad- code markup in comments should work. Look at the ? near "Write your comment" and you will see some help. In particular, you can use tags pre and / pre inside angle brackets - though it won't colorize. And it would be up to you to provide the indentation. --Loren
Dave Bergstein replied on : 9 of 17
@Brad: I should have clicked the help! Here's the code again formatted:
myclass.m:
classdef myclass
    properties
        Prop1 = 20;
    end

    methods
        function out = method1(obj)
            out = obj.Prop1 * 10;
        end
    end
end

perftest.m:
m = myclass;
for i = 1:10^5
    method1(m);
end
@Michal: Assessing performance can be complicated. This is why we turned to customer applications we collected to create the histogram we show. I'm encouraging a colleague of mine to write another blog post on the MATLAB Profiler. Though not a perfect reflection of performance, it's a great tool for finding areas to optimize. And as our JIT compiler technology advances, MATLAB can increasingly do more optimization for you. Btw, if you have specific code examples you are concerned about, please share them with technical support. Cases (including identified anomalies) help us as we continue to improve performance.
Brad Stiritz replied on : 10 of 17
Thanks, Loren. Apologies for "deducing" on 1-sample evidence (embarrassing!) that code formatting might not be supported :\
Loren Shure replied on : 11 of 17
Actually we are having trouble correctly displaying code formatting in comments - Dave fixed his comments, but whitespace is being violated now. I have brought this to the attention of someone who should be able to fix it. --Loren
Sebastian replied on : 12 of 17
I've been using Matlab for a long time and it is one of my favorite computational software. It's great to see that Matlab is improved all the time.
Michal Kvamicka replied on : 13 of 17
The problem with deformed performance results with and without profiler described here was not solved in R2016a. So, the profiling with new JIT engine is still highly unreliable!!! Let me know how should users use MATLAB profiler in case when MATLAB produce very contradictory results?
Marios replied on : 14 of 17
Hi Loren. I was also pretty disappointed to see that a big, high-performance (vectorized etc.) part of our codebase (pretty much all OO) experienced on the average a 20% slowdown going from r2015a to r2015b. After spending quite a bit of time in the profiler I still can't figure a pattern on how to improve things. At this point my only solution is to skip r2015b and wait until r2016a to see if that changes anything. Just my honest feedback on the release.
Brett Howell replied on : 15 of 17
This JIT change has drastically reduced the speed of our code. We are working with very large numbers of differential equations and very large numbers of variables within those equations, and the code can't be practically broken up. Before 2015b, we could declare over 1000 variables in a parent function and access those variables in a nested function with minimal memory problems. The new JIT change makes our code take 10-30 times longer to run, and no one at matlab seems to have an answer. We produce a fairly widely used application, and the update is having a broad negative impact.
Bogdan Timofte replied on : 16 of 17
Speed indeed is crucial. Without optimizations for speed MATLAB would become useless.
Dave Bergstein replied on : 17 of 17
@Sebastian: Thanks for the positive feedback. @Michal, @Marios, and @Brett: Thank you for your input. We’re following up with you offline to learn more about your specific cases. Also, for those interested, Yair’s book “Accelerating MATLAB Performance” walks through some examples of identifying and optimizing performance bottlenecks. The MATLAB Profiler isn’t a perfect reflection of performance. But it's a tool that can point you to areas to explore further.