{"id":2750,"date":"2024-09-26T07:24:00","date_gmt":"2024-09-26T11:24:00","guid":{"rendered":"https:\/\/blogs.mathworks.com\/matlab\/?p=2750"},"modified":"2024-09-26T07:24:00","modified_gmt":"2024-09-26T11:24:00","slug":"matlab-now-has-over-1000-functions-that-just-work-on-nvidia-gpus","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/matlab\/2024\/09\/26\/matlab-now-has-over-1000-functions-that-just-work-on-nvidia-gpus\/","title":{"rendered":"MATLAB now has over 1,000 functions that Just Work on NVIDIA GPUs"},"content":{"rendered":"

GPU support in MATLAB started in R2010b<\/span><\/h2>
Back in R2010b, the first GPU enabled functions were made available in MATLAB via <\/span>Parallel Computing Toolbox<\/span><\/a>. The idea was then, as it is now, to overload existing MATLAB functions such that they accept the <\/span>gpuArray<\/span><\/a> type. If you gave a <\/span>gpuArray<\/span> to a function then it would automatically work on the GPU without the user having to do anything else. That is, if you had MATLAB code like this<\/span><\/div>
y = fft(x); <\/span>% Compute the fast fourier transform of x on the CPU<\/span><\/span><\/div><\/div><\/div>
Then all you need to do to run that on the GPU is use <\/span>gpuArray<\/span> and <\/span>gather<\/span><\/a> to take care of the transfer to and from the GPU. Other than that, the MATLAB code is the same<\/span><\/div>
gpuX = gpuArray(x); <\/span>% Transfer the array x to the GPU<\/span><\/span><\/div><\/div>
 <\/div><\/div>
gpuY = fft(gpuX); <\/span>% fft is now performed on the GPU<\/span><\/span><\/div><\/div>
 <\/div><\/div>
y = gather(gpuY); <\/span>% Gather the result from the GPU<\/span><\/span><\/div><\/div><\/div>
R2010b provided support for 123 such functions, most of which were element-wise math such as <\/span>sin<\/span>, <\/span>exp<\/span> along with arithmetic operators such as <\/span>+<\/span> and <\/span>-.<\/span> There were also some more interesting things such as <\/span>fft<\/span><\/a>,<\/span> <\/span>mtimes<\/span><\/a> (matrix-matrix multiplication) and the all important <\/span>mldivide<\/span><\/a>, the full name for backslash, <\/span>possibly the most famous operator in MATLAB<\/span><\/a>.<\/span><\/div>
It was a great start but there were also some awkward omissions, the most glaring of which were <\/span>subsref<\/span> and <\/span>subsasgn<\/span>. This meant indexing of <\/span>gpuArray<\/span>'s was not supported. Imagine trying to do anything useful in MATLAB without ever indexing anything! My colleagues tell me that this made for an interesting trip to the SC 2010 super-computing conference where they introduced MATLAB's GPU functionality for the first time.<\/span><\/div>

As of R2024b, 1195 MATLAB functions have gpuArray support<\/span><\/h2>
14 years later and things have changed drastically. As you browse the MATLAB documentation, you'll note that many functions have an <\/span>Extended Capabilities <\/span>section. Here's that section for <\/span>fft<\/span><\/a> in MATLAB R2024b where you can see that there are not just one but two entries related to GPUs: <\/span>GPU Code Generation<\/span> and <\/span>GPU Arrays<\/span>. We'll talk about GPU code Generation another time. <\/span><\/div>
\"Screenshot<\/img><\/div>
Today, we are focused on GPU Arrays, the topic of today's post. In R2024b. <\/span>1195 functions across 14 toolboxes have now got support for <\/span>gpuArray<\/span>, which is a lot!<\/span><\/div>
The chart below shows this growth over time.<\/span><\/div>
\"\"<\/img><\/div>
Working with <\/span>gpuArray<\/span> in this way can be extremely effective. One <\/span>public case study I can point to is from NASA's Langley Research Center<\/span><\/a> who tell us that it took them 30 minutes to get their MATLAB algorithm working on the GPU resulting in a 40x speedup. <\/span><\/div>

What does gpuArray support really mean?<\/span><\/h2>
When you start digging down into the details, what we mean by 'support for <\/span>gpuArray'<\/span> is both really simple and full of complications. At the simple end of the spectrum, it means that the function in question can accept <\/span>gpuArray<\/span>'s and do something with them. The complexity starts to creep in when we ask what that 'something' is.<\/span><\/div>
Of the 1195 functions with <\/span>gpuArray<\/span> support, <\/span>729 of them are supported with no limitations<\/span>. Anything you can do on the CPU with a normal MATLAB array, you can do on the GPU with a <\/span>gpuArray<\/span>. This can make it incredibly easy to switch from using the CPU to the GPU for quite a lot of MATLAB code.<\/span><\/div>
The other<\/span> 466 functions have some kind of restriction or caveat<\/span>. The <\/span>fft<\/span> function is one of these. Expand the <\/span>GPU Arrays<\/span> section in <\/span>Extended Capabilities<\/span> and you'll see what they are.<\/span><\/div>
\"\"<\/img><\/div>
Many of these caveats are minor. I don't consider the detail for <\/span>fft<\/span> to be a big deal, although if you disagree do let me know and why. Other restrictions might be more problematic. The restriction for the <\/span>lu<\/span> function<\/span><\/a>, for example, is that it doesn't accept sparse <\/span>gpuArray<\/span>. The <\/span>inv<\/span> function<\/span><\/a> also doesn't accept sparse <\/span>gpuArray<\/span> and you also have to be more careful when dealing with badly scaled or nearly singular matrices since the <\/span>gpuArray<\/span> version of <\/span>inv<\/span> won't warn you when you have them whereas the standard version will. Even <\/span>mtimes<\/span>, the full name of MATLAB's matrix-matrix multiplication operator has a restriction: it doesn't support <\/span>int64<\/span><\/a> on the GPU.<\/span><\/div>
You may reasonably ask \"Why do these restrictions and caveats exist?\". Indeed, the reviewer of this blog post asked exactly that! Often, it's for performance reasons. Take <\/span>fft<\/span>, for example. To <\/span>know<\/span> that the result has all zero imaginary part we'd have to run another GPU kernel to go and check all the values. We would then need to wait for this operation to finish (i.e. synchronize the device) to see what the answer is. If the imaginary parts are all zero, we would then need to run another kernel to de-interlace the array and drop the imaginary part. On a GPU, this could result in significant slowdown and yet most of the time the imaginary parts are non-zero. That is, <\/span>everything would be made much slower for a convenience that's rarely needed<\/span>. Of course some restrictions are there simply because it would take a lot of effort to implement them and we haven't seen the demand yet. As always, your feedback is essential here! <\/span><\/div>
At the other end of the scale we have around <\/span>140-150 functions that accept a GPU array but don't actually run on the GPU<\/span>! These are essentially convenience functions such as <\/span>plot<\/span> that have been modified so that they Just Work when you send them a <\/span>gpuArray<\/span>. This means that, for example, you can do <\/span><\/div>
x = gpuArray.linspace(-pi,pi,1000); <\/span>% Construct x directly on the GPU<\/span><\/span><\/div><\/div>
y = sin(x); <\/span>% Compute sin(x) on the GPU<\/span><\/span><\/div><\/div>
plot(x,y); <\/span>% produce the plot<\/span><\/span><\/div><\/div><\/div>
Instead of <\/span><\/div>
x = gpuArray.linspace(-pi,pi,1000); <\/span>% Construct x directly on the GPU<\/span><\/span><\/div><\/div>
y = sin(x); <\/span>% Compute sin(x) on the GPU<\/span><\/span><\/div><\/div>
CPUx = gather(x); <\/span>% Bring x from GPU into main memory<\/span><\/span><\/div><\/div>
CPUy = gather(y); <\/span>% Bring y from GPU into main memory<\/span><\/span><\/div><\/div>
plot(CPUx,CPUy); <\/span>% Do the plot<\/span><\/span><\/div><\/div><\/div>
When you send <\/span>gpuArray<\/span>s to <\/span>plot<\/span>, the <\/span>gather<\/span> operations are done for you behind the scenes. So, <\/span>plot<\/span> supports <\/span>gpuArray<\/span> even though the plot operation is not done by the GPU.<\/span><\/div>

Pagewise backslash - the 1000th function to support gpuArray<\/span><\/h2>
While gathering the numbers for this post, a few of us got fixated on what the 1000th function might be. It turns out to be slightly tricky to figure this out but we are reasonably confident that the 1000th MATLAB function to support <\/span>gpuArray<\/span> is <\/span>pagemldivide<\/span><\/a> -- the pagewise version of <\/span>mldivide<\/span><\/a>, known more commonly as the backslash operator. That the 1000th <\/span>gpuArray<\/span> function is directly related the most iconic of MATLAB functions, and one of the first functions that ever received <\/span>gpuArray<\/span> support, seems rather fitting.<\/span><\/div>

Moving beyond gpuArray<\/span><\/h2>
gpuArray<\/span> is the easiest way of getting many applications ported to use the GPU but there's more to GPUs in MATLAB than <\/span>gpuArray<\/span>. Here are some pointers to where you might go next<\/span><\/div>