Loren on the Art of MATLAB

Controlling Multithreading 31

Posted by Loren Shure,

With Release R2007b, you now have the ability to set (and get) the maximum number of allowed threads. What does this mean for you?

Contents

The Skinny on maxNumCompThreads

Here's some pertinent information about maxNumCompThreads.

  • maxNumCompThreads allows you to programmatically control the maximum number of computational threads that MATLAB uses.
  • Unlike the setting from the preferences panel, this setting does not persist between MATLAB sessions.
  • maxNumCompThreads controls both the JIT and BLAS, meaning both element-wise and matrix operations. Since the JIT doesn't operate at the command line, maxNumCompThreads only affects element-wise computations performed from M-files.
  • The 'automatic' setting is designed to take best advantage of the hardware the program is currently running on.
  • Not all operations run with the maximum number of computational threads. When MATLAB uses fewer than the maximum, it's avoiding the overhead of threading which can dominate certain operations, for example, matrix multiplication for 2x2 matrices.
  • Currently, only a subset of functions that could benefit from multithreading are taking advantage of it. We will expand this list with each upcoming release.

When to Use maxNumCompThreads

The main message here is that you should not generally need to use this function at all! Why? Because we'd like to make MATLAB do the best job possible for you. However, there are some situations where the use of maxNumCompThreads is merited.

  • Benchmarking - This needs to be done with care. For example, as I mentioned earlier, matrix multiplication doesn't use multiple threads for 2x2 matrices since the thread overhead dominates the numerical work that's being done. Be sure to test a variety of sizes to characterize your results.
  • If you notice some slowdown in code that was previously running faster, you can programmatically reduce maxNumCompThreads and then restore the maxNumCompThreads value when the calculation is finished.

If You Find a Performance Issue...

Please let us know if you encounter a situation in which setting maxNumCompThreads manually is necessary for better performance, by contacting Technical Support. It is our intention that you should not need to use maxNumCompThreads and it will help us to improve the code if you notify us of such situations.


Get the MATLAB code

Published with MATLAB® 7.5

31 CommentsOldest to Newest

Thanks to some folks at The MathWorks, I have some additional links with information available for you. Here is a technical note describing which MATLAB functions currently support multi-threaded computations. And the documentation explains more of the details.

Loren I think the new Matlab will be great and I can’t wait to receive our new version. However, I proposed the change earlier, will we be able to inherent built-in structures in this version of Matlab?
Thanks!

I ask this because writting low-level methods like ‘display’, ‘subsref’, ‘subsasgn’ is too much pain for some objects which should simply behave like a structure in Matlab, with some additional new methods. I feel many people don’t want to utilize the Object Oriented feature of Matlab simply because they can’t use those built-in methods. Will this be allowed in any forseeable timeframe?
Sorry to bother and thanks.

Bruce-

I am not sure why you posted this question on this topic as I see not direct relationship. In R2007b, you can not inherit directly from a struct in MATLAB.

–Loren

Sorry about this. I wanted to delete my post but I couldn’t. If you could pls just take them off. Originally, I want to ask if Distributed Computing engine licenses are required for dual quad core machines. But then I switched to the question I’m concerned about most. Sorry again!

Hello, while I know that the intention is for people not to write code that specifically tries to use this multi-threading option, I am wondering if there is a way to make code which is fairly parellelizable run on different threads.

I’ll give more detail. I, as I’m sure many others, use MATLAB to run many Monty Carlo Simulations, which basically involve running the same code over and over again for different seed values. Before I updated my version of matlab I’d take advantage of the multi cores on my machine by simply opening up multiple versions of matlab and running the same program several times. This is of course a ugly workaround. However I found that this workaround still runs faster than if I have multithreading turned on. Any comments/hints on how I can more elegently run these scenarios would be appreciated.

Aaron-

Unfortunately, so much depends on the exact code. I recommend you contact technical support with your specific set up and code details and they can help guide you.

–Loren

I have several related but distinct real-time monitoring tasks. It would be cleanest to code them as separate files, and run each in a Matlab instance under Linux, but I expect that would cost me both in context switching and memory.

Alternative 1, run each of those m files as a thread under one Matlab. That should be lighter weight, but according to the Matlab 6 release notes, quote
On Linux, you may notice that MATLAB can start up a large number of processes, even to handle simple tasks. These are actually threads, but due to the “one process per thread” model on Linux, they appear as processes. When you use the ps or top commands in Linux to show the processes running on your system, you will see each of these threads listed as a separate process.

Alternative 2, all the monitoring tasks in one big ugly loop
is probably the most efficient.

Advice?

I use 32-bit MATLAB 2007a on a 64-bit windows XP machine equipped with 4 GB RAM and quad core processor. I have enabled multithreading in MATLAB preferences dialog so that all 4 processors are used. I have implemented a function which resembles convolution of matrices but not exactly. Now the interesting bit is that MATLAB seems to use all 4 processors (when I monitor them in task manager), but at mutually exclusive times. i.e. when one processors activity graph is up the rest three are down with total processor utilization hovering around 25%. Why is it so? How could I make MATLAB use all 4 cores simultaneously…

The code roughly is like this
for i=1:n
for j=1:m
produce shifted copy of 200×200 matrices
multiply above two and one more matrix with .*
val(m,n)= sum(sum(result of multiplication))
end
end

First R2008a is released and the multithreaded computation is on by default.

The multithreaded computation in MATLAB falls into two catergories. First the elementwise operations like +, .^, via JIT and second linear algebra operations like *, \. via the use of BLAS libraries. Both have their own limitations and characteristics.

However, having multithreaded computation turned on does not mean that all computation will be done in multiple threads. It is because there is cost associated with the creation and maintainence of threads.

In general, MATLAB will base on some heuristic to determine whether the computation will be done in multiple threads.

To understand whether an computation task can be benefited from multithreading, you need to know the ratio between the amount of memory needed/used (says number of elements in a matrix) and the number of floating point operations (flops) once the memory gets into cache/memory.

Given n-by-n matrices A and B, A+B needs to move n^2 elements and perform n flops. So you are doing 1/2n flop per element of the matrix. Since + is really fast and the bottleneck is in moving memory, you will not see multithreading with + in MATLAB.

Consider A*B, there are 2n^2 elements and 2n^3 flops. You are doing n flops per element. You will see benefit of multithreading if n is large enough.

Looking at your example, I think it is similar to A+B and as a result you won’t see the benefit of multithreading.

It does look like each iteration of the for loop are independent. We don’t have parallel for support in R2007a within MATLAB. But you may want to check up PARFOR support in the latest release with the Parallel Computing toolbox.

Thanks very much for explanation Bobby. The operations are indeed ‘per element’ and loops are independent. I hope my uni gets 2008a soon.

I was wondering whether there was an example code that uses multithreading in matlab.

Say we have 2 processes, process1 and process2.
If I write :

resultOfProcess1 = process1();
resultOfProcess2 = process2();

Does matlab automatically run two separate threads for these applications?

What if there are 100 such processes? Is there any example code that shows how to use matlab to spawn 100 processes, wait for their results, and then repeat?

Is it something like:

for i = 1:100
process(i);
end

??

Thanks.

Serkan-

You should really read the documentation. That’s now how the multithreading works in MATLAB today. If you have multithreading on, then when you execute a statement, it will try to run in more than 1 thread, should that make sense and be available to the running command. You can’t specify what thread to run on programmatically nor can you run more than 1 statement at a time.

If you want to batch up code to run in multiple processes, you need the Parallel Computing Toolbox and possibly the MATLAB Distributed Computing Server.

–Loren

I have a compiled matlab application for high frequency trading, which is set to use multiple threads. The program is compiled into Java and is called by Java for each update cycle. But inside MATLAB, there are multiple periodic Timers. My question in this case is:

Is each timer running on separate threads?

Thanks. (Matlab 2007b)

Yunde-

As far as I know, currently all the timers run in one shared thread. Since that is something under the hood, it could change in a future release.

–Loren

Dear Loren,

I’m a little late to this party, but hopefully you or your readers can provide some feedback. I have some code that iteratively builds an n-D array. On my PCs (2-core and 4-core) R2008a’s multithreading utilizes all cores to run the code. I recently tried the same code on a 24-processor workstation. While maxNumCompThreads returns 24, only one CPU is active. Another commenter asked about this in one of your previous posts, but I didn’t see any responses.

Would anyone care to chime in with insights?

Thanks,
J. Hengenius

J Hengenius-

Whether or not MATLAB uses more than 1 thread for an operations depends very much on what operation is being done. You haven’t supplied enough information for any of us to give a concrete answer, including how you know the 2- and 4- core PCs are using the multithreading. Perhaps you can send the details to tech support for more context.

–Loren

Thanks for the quick response. I’ve been using MS Task Manager to check my PC cores. All cores run at 100% while the code is running.

The n-D grid created by the code represents the parameter space of an ODE system. At each point in the parameter grid, a cost function (ODE output vs. data) is evaluated and stored. There is a heuristic to decide if the resolution along any one parameter dimension is high enough. If it isn’t, the number of points along that dimension is doubled and the cost function is evaluated/stored at the new points. The evaluations don’t depend on each other and these evaluations are carried out simultaneously on all cores of my PCs (maxNumCompThreads returns 2 or 4 depending on the PC).

The multi-processor system has been executing the same code on only one processor. (It may be worth noting that my PCs are running Vista. The workstation has basic SPARC architecture and is running Solaris.)

If this is getting outside your area of expertise, I’ll pass it along to Technical Support. I’d still welcome feedback from you or your commenters. Thanks again.

- J. Hengenius

J. Hengenius-

You still haven’t shown code that we could look at. Please send a sample to technical support with information on the runs on different computers so they can dive into this with more information.

–Loren

I’m running a home made application using Agilent VEE Pro 8.5 with has embeded Matlab 2007a. When the application reach a Matlab script (show below) it slow down (average 9 loops per second) . I’ve been monitoring the processor with Task Manager (Windows Vista Ultimate 32 bits, HP Pavilion a1710n with ADM Athlon 64 X2 Dual Core Processor 4200+ 2.20 GHz) and I can see that only one of the core is working. I run the same application using Agilent VEE Pro 6.0 with has embeded Matlab 5.3 on an MS Windows XP PC without dual core and the performace is a lot higher (average 700 loops per second). I’ve try to find information about how to programatically active multithreading with the embeded version of matlab but I could not find any information. But even, only with one core working, the has decrease considerable. Could you help me to solve this problem? What can I do to reach the desired performance (same as with XP)?
ZI = 0:1:C;
W = polifit(A,B,4);
Vout = polycal (W, ZI);
Z1 = ZI

Thanks,
Uvernes

Uvernes-

For a start, MATLAB wasn’t using multi-threaded BLAS in version 5.3. That has come much more recently.

You probably should contact technical support as your issue warrants more attention than I can give it here.

–Loren

Automatic multithreading is great when you’re running on a multicore workstation (i.e., dedicated to a single user), but what about the situation when multiple users are sharing a multiprocessor system? The problem we’re running into is that the available processors are being oversubscribed, resulting in lost performance due to OS scheduling overheads, cache pollution, memory contention, etc.

Is there some way to change the multithreading default at a system-wide level from “automatic” to “manual” so that we (the system administrators) can keep the aggregate thread count to something manageable?

Loren, Tom-

You can set maxNumCompThreads(1); in toolbox/local/matlabrc.m, thus overwriting any user setting.
You can do this in 2 ways:

a) add the line before the section where user preferences are checked. In this way the preferences of the users are still repected, but the internal default behavior is overwritten.

b) add the line after the section. In this way you overwrite both the user settings and the internal setting, and you force your users to activate multi-threading explicitly in every session. That’s how we do it here.

The new default, i.e. automatic multi-threading, plus the problem with the Java garbage collector threads (1 per core) can turn Matlab into a DoS tool on our big SMPs, with 96-144 cores!

-Bernd

Hello All

I second Tom Crocket’s comment.

Inside a matlab script (batch mode) I tried

LASTNT=maxNumCompThreads(NTHREADS)

with several values for NTHREADS (1,4,8, etc)
on a 8-core 64bit Linux machine.
However, “top” shows all 8 cores working at ~100%,
regardless of the value I use for NTHREADS.

I tried also with Matlab interactively,
but I get the same 8 cores working, no matter what I do.

(Matlab R2009a_64bit, Linux, CentOS 5.2)

Is there a way to limit the number of cores in use?
That would be very important and useful for multi-user
systems.

Thank you,
Gus Correa

Gus-

Multithreading features change in MATLAB each release recently. I suggest reading release notes and contacting technical support (link at right) for the best information for your installation and configuration.

–Loren

Hi Loren,

here is a good question on MATLAB timers:
Suppose I want to update several variables (say a and b) “atomically” in MATLAB (for the purposes of multiple timers).

Would [a,b] = deal(…,…); do the job, or can the assignment be interrupted by a timer? If it is safe in current versions (which seems very likely to me), it is probably a good idea to retain this behavior for future MATLABs (potentially as valuable as having a compare-and-swap instruction in a processor).

Thanks,
Christian

Christian-

I have no idea about your question. You can either contact technical support or set up a test of your own where the assignment takes long enough that you can test if it gets interrupted. Also, you should probably make this an enhancement request – please see the support link on the right of this blog.

–Loren

These postings are the author's and don't necessarily represent the opinions of MathWorks.