{"id":230,"date":"2022-05-19T09:40:07","date_gmt":"2022-05-19T13:40:07","guid":{"rendered":"https:\/\/blogs.mathworks.com\/matlab\/?p=230"},"modified":"2022-05-19T09:44:32","modified_gmt":"2022-05-19T13:44:32","slug":"how-to-make-a-gpu-version-of-this-matlab-program-by-changing-two-lines","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/matlab\/2022\/05\/19\/how-to-make-a-gpu-version-of-this-matlab-program-by-changing-two-lines\/","title":{"rendered":"How to make a GPU version of this MATLAB program by changing two lines"},"content":{"rendered":"
In his article, <\/span>A short game of Life<\/span><\/a>, Steve Eddins showed us the following few lines of code that impemented <\/span>Conway's game of life<\/span><\/a>. Steve's version used a 750 x 750 gameboard whereas mine is using 2000 x 2000 because I want something meaty to compute<\/span><\/div>
clear <\/span>% Clear all variables<\/span><\/span><\/div><\/div>
 <\/div><\/div>
tic<\/span><\/span><\/div><\/div>
im = rand(2000,2000)>0.8;<\/span><\/span><\/div><\/div>
for <\/span>k=1:500<\/span><\/span><\/div><\/div>
t = conv2(im(:,:,k),[2,2,2;2,1,2;2,2,2],<\/span>\"same\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,k+1) = (t > 4) & (t < 8);<\/span><\/span><\/div><\/div>
end<\/span><\/span><\/div><\/div>
OriginalTime = toc<\/span><\/span><\/div>
OriginalTime = 39.4715<\/div><\/div><\/div><\/div>
As Steve showed us, this is extremely easy to turn into an animated gif. Since my gameboard is so big, I'll just zoom in on a small section of it <\/span><\/div>
imwrite(~im(1:200,1:200,1,:),<\/span>\"Life.gif\"<\/span>,<\/span>\"DelayTime\"<\/span>,0.02,<\/span>\"LoopCount\"<\/span>,Inf)<\/span><\/span><\/div><\/div><\/div>
\"life.gif\"<\/img><\/div>
The first step to faster code: Preallocation<\/span><\/div>
I'm obsessed with speed in the MATLAB programming language and wondered if there is anything that can be done with these few lines to speed them up without ruining their elegance too much.<\/span><\/div>
Let's start with something easy: preallocation of the output arrays, <\/span>im<\/span><\/div>
clear <\/span>% Clear all variables<\/span><\/span><\/div><\/div>
 <\/div><\/div>
tic<\/span><\/span><\/div><\/div>
im = zeros(2000,2000,1,501,<\/span>\"logical\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,1) = rand(2000,2000) > 0.8;<\/span><\/span><\/div><\/div>
for <\/span>k=1:500<\/span><\/span><\/div><\/div>
t = conv2(im(:,:,k),[2,2,2;2,1,2;2,2,2],<\/span>\"same\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,k+1) = (t > 4) & (t < 8);<\/span><\/span><\/div><\/div>
end<\/span><\/span><\/div><\/div>
PreallocatedTime = toc<\/span><\/span><\/div>
PreallocatedTime = 14.2942<\/div><\/div><\/div><\/div>
Over 2x speed up for one extra line and one modified line. Not bad but can I go any further?<\/span><\/div>
Moving from a CPU implementation to a GPU implementation of many MATLAB codes is very easy!<\/span><\/div>
My computer has quite a nice NVIDIA GPU which I can access using <\/span>Parallel Computing Toolbox<\/span><\/a><\/div>
gpuDevice()<\/span><\/span><\/div>
ans = <\/span><\/div>
CUDADevice with properties:\r\n\r\n Name: 'NVIDIA GeForce RTX 3070'\r\n Index: 1\r\n ComputeCapability: '8.6'\r\n SupportsDouble: 1\r\n DriverVersion: 11.6000\r\n ToolkitVersion: 11.2000\r\n MaxThreadsPerBlock: 1024\r\n MaxShmemPerBlock: 49152\r\n MaxThreadBlockSize: [1024 1024 64]\r\n MaxGridSize: [2.1475e+09 65535 65535]\r\n SIMDWidth: 32\r\n TotalMemory: 8.5894e+09\r\n AvailableMemory: 7.2939e+09\r\n MultiprocessorCount: 46\r\n ClockRateKHz: 1725000\r\n ComputeMode: 'Default'\r\n GPUOverlapsTransfers: 1\r\n KernelExecutionTimeout: 1\r\n CanMapHostMemory: 1\r\n DeviceSupported: 1\r\n DeviceAvailable: 1\r\n DeviceSelected: 1\r\n<\/div><\/div><\/div><\/div><\/div><\/div>
GPUs are perfectly suited to this kind of thing but writing a GPU version of this would be difficult right? Late nights of hardcore CUDA coding await. <\/span><\/div>
But there's another way! <\/span><\/div>
Well over <\/span>1000 MATLAB functions<\/span><\/a> (including those in the official Toolboxes) are 'overloaded' with the <\/span>gpuArray<\/span> data type. What this means in practice is that whereas this code runs on the CPU:<\/span><\/div>
A = rand(3);<\/span><\/span><\/div><\/div>
B = rand(4);<\/span><\/span><\/div><\/div>
Cfull = conv2(A,B)<\/span><\/span><\/div>
Cfull = <\/span>6\u00d76<\/span><\/div><\/div>
0.2357 0.1997 0.7748 0.4111 0.5196 0.0080\r\n 0.2237 0.4145 1.3158 1.0064 1.2236 0.0201\r\n 0.6943 0.7811 2.3386 1.8777 1.5679 0.0412\r\n 0.3726 0.9340 2.2665 2.3125 1.7186 0.5522\r\n 0.5210 0.8183 1.7277 1.9970 1.4909 0.8490\r\n 0.2133 0.8088 1.2790 1.8410 1.2599 0.5397\r\n<\/div>
<\/div>
<\/div><\/div><\/div>
<\/div>
<\/div>
<\/div><\/div><\/div><\/div><\/div>
This code runs on the GPU:<\/span><\/div>
gpuA = gpuArray(A); <\/span>% Transfer A to the GPU and call it gpuA<\/span><\/span><\/div><\/div>
gpuB = gpuArray(B); <\/span>% Transfer B to the GPU and call it gpuB<\/span><\/span><\/div><\/div>
Cfull_gpu = conv2(gpuA,gpuB) <\/span>% This now runs on the GPU <\/span><\/span><\/div>
Cfull_gpu =\r\n\r\n 0.2357 0.1997 0.7748 0.4111 0.5196 0.0080\r\n 0.2237 0.4145 1.3158 1.0064 1.2236 0.0201\r\n 0.6943 0.7811 2.3386 1.8777 1.5679 0.0412\r\n 0.3726 0.9340 2.2665 2.3125 1.7186 0.5522\r\n 0.5210 0.8183 1.7277 1.9970 1.4909 0.8490\r\n 0.2133 0.8088 1.2790 1.8410 1.2599 0.5397<\/div><\/div><\/div><\/div><\/div>
All you need to do in order to make over 1000 MATLAB functions work on an NVIDIA GPU is Parallel Computing Toolbox which allows you to give them <\/span>gpuArray<\/span>s instead of normal arrays. <\/span><\/div>
Whether or not you'll actually get a speed up depends on many factors but to get started, to simply get things running on the GPU instead of the CPU, this is it!<\/span><\/div>
Back to Steve's code. What I need to do is change the initialisation of <\/span>im<\/span> <\/span>to a <\/span>gpuArray<\/span> and everything will automagically run on the GPU. That is I change<\/span><\/div>
im = zeros(2000,2000,1,501,<\/span>\"logical\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,1) = rand(2000,2000) > 0.8;<\/span><\/span><\/div><\/div><\/div>
to<\/span><\/div>
im = zeros(2000,2000,1,501,<\/span>\"logical\"<\/span>,<\/span>\"gpuArray\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,1) = rand(2000,2000,<\/span>\"gpuArray\"<\/span>) > 0.8;<\/span><\/span><\/div><\/div><\/div>
Let's give it a try<\/span><\/div>
clear <\/span>% Clear all variables<\/span><\/span><\/div><\/div>
 <\/div><\/div>
dev = gpuDevice();<\/span><\/span><\/div><\/div>
tic<\/span><\/span><\/div><\/div>
im = zeros(2000,2000,1,501,<\/span>\"logical\"<\/span>,<\/span>\"gpuArray\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,1) = rand(2000,2000,<\/span>\"gpuArray\"<\/span>) > 0.8;<\/span><\/span><\/div><\/div>
for <\/span>k=1:500<\/span><\/span><\/div><\/div>
t = conv2(im(:,:,k),[2,2,2;2,1,2;2,2,2],<\/span>\"same\"<\/span>);<\/span><\/span><\/div><\/div>
im(:,:,1,k+1) = (t > 4) & (t < 8);<\/span><\/span><\/div><\/div>
end<\/span><\/span><\/div><\/div>
wait(dev);<\/span><\/span><\/div><\/div>
GpuTime = toc<\/span><\/span><\/div>
GpuTime = 5.3013<\/div><\/div><\/div><\/div>
Almost 3x faster than the CPU version that used preallocated arrays and around 7x faster than the original! Not bad considering I only changed 2 lines of code. Furthermore, this has to be the easiest GPU 'port' of a simulation I've ever written<\/span><\/div>
Now, I am sure that there are CUDA experts out there who could do better than this -- squeezing every last drop of performance possible from the poor overworked GPU -- but 3x speedup for so little work is pretty good going and there are several options in the MATLAB ecosystem that allow you to go deeper and explore other approaches.<\/span><\/div>

What's going on with wait(dev)?<\/span><\/h3>
The eagle-eyed among you might have noticed that my GPU version has an extra couple of lines in it that I haven't mentioned yet. As we speak, I can feel you reaching for the comment button to tell me that I lied to you....I changed four lines not two! Here are the two lines I conveniently chose not to mention to you<\/span><\/div>
\"\"<\/img><\/div>
The reason for these lines is all about timing. You see, when you ask for a computation to be done on the GPU, MATLAB kicks things off and moves to the next line <\/span>without waiting for the GPU to finish the calculation. <\/span>This can be used for some very nifty interleaving of code, where you have things running on the CPU and GPU simultaneously, but it can also mess up timing if you use <\/span>tic\/toc<\/span>. Timing GPU code can be tricky which is why MathWorks also give you the <\/span>gputimeit<\/span><\/a> command.<\/span><\/div>
If all I wanted to do was run the code, and not time it, then I wouldn't need to bother with these two extra lines. So what I told you is true... from a certain point of view.<\/span><\/div>

System details<\/span><\/h3>
  • MATLAB R2022a<\/span><\/li>
  • CPU: 11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz<\/span><\/li>
  • GPU: NVIDIA GeForce RTX 3070<\/span><\/li>
  • OS: Windows 11<\/span><\/li><\/ul>

    Over to you<\/span><\/h3>
    Have you got some code that can be easily made to run on a GPU like this and show a performance boost? <\/span><\/div>
    <\/div>\r\n<\/div>