{"id":927,"date":"2013-12-10T13:19:47","date_gmt":"2013-12-10T18:19:47","guid":{"rendered":"https:\/\/blogs.mathworks.com\/steve\/?p=927"},"modified":"2019-11-01T09:38:56","modified_gmt":"2019-11-01T13:38:56","slug":"image-processing-with-a-gpu","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/steve\/2013\/12\/10\/image-processing-with-a-gpu\/","title":{"rendered":"Image processing with a GPU"},"content":{"rendered":"
\r\n

\r\n\r\nI'd like to welcome guest blogger Anand Raja for today's post. Anand is a developer on the Image Processing Toolbox team. -Steve\r\n<\/em>\r\n<\/p>\r\n

Many desktop computers and laptops now come with fairly powerful Graphics Processing Units (GPU's). Initially, GPU's were mostly used to power computations for graphics applications, but soon people realized that they are just as useful for any kind of numerical computing.<\/p>

GPU's are made of a large number of processing units which by themselves aren't very powerful, but become formidable when used in tandem. So, if you have processing to be done that is parallelizable, the GPU will be a great fit.<\/p>

With that in mind, isn't it almost obvious that image processing is a great fit for GPU's! A lot of image processing algorithms are data-parallel, meaning the same task\/computation needs to be performed on many elements of the data. Lots of image processing algorithms either operate on pixels independantly or rely only on a neighborhood around pixels (like image filtering).<\/p>

So, lets get down to it. My desktop computer has a GPU, and I want to do some image processing using my favorite software (no prizes for guessing), MATLAB. Note that in order to interact with the GPU from MATLAB, you require the Parallel Computing Toolbox<\/a>.<\/p>

I can use the gpuDevice<\/a> function to get information about my GPU.<\/p>

gpuDevice\r\n<\/pre>
\r\nans = \r\n\r\n  CUDADevice with properties:\r\n\r\n                      Name: 'Tesla C2075'\r\n                     Index: 1\r\n         ComputeCapability: '2.0'\r\n            SupportsDouble: 1\r\n             DriverVersion: 5.5000\r\n            ToolkitVersion: 5\r\n        MaxThreadsPerBlock: 1024\r\n          MaxShmemPerBlock: 49152\r\n        MaxThreadBlockSize: [1024 1024 64]\r\n               MaxGridSize: [65535 65535 65535]\r\n                 SIMDWidth: 32\r\n               TotalMemory: 5.6368e+09\r\n                FreeMemory: 5.5362e+09\r\n       MultiprocessorCount: 14\r\n              ClockRateKHz: 1147000\r\n               ComputeMode: 'Default'\r\n      GPUOverlapsTransfers: 1\r\n    KernelExecutionTimeout: 0\r\n          CanMapHostMemory: 1\r\n           DeviceSupported: 1\r\n            DeviceSelected: 1\r\n\r\n<\/pre>
Seeing that I have a supported GPU, I can read an image and transfer the image data to my GPU using the constructor for the gpuArray<\/a> class. The gpuArray object is used to access and work with data on the GPU.<\/p>
im = imread('concordaerial.png'<\/span>);\r\nimGPU = gpuArray(im);\r\nimshow(imGPU);\r\n<\/pre>\"\" 
So imGPU is a gpuArray object containing data of type uint8.<\/p>
class(imGPU)\r\nclassUnderlying(imGPU)\r\n<\/pre>
\r\nans =\r\n\r\ngpuArray\r\n\r\n\r\nans =\r\n\r\nuint8\r\n\r\n<\/pre>
A number of the functions in the Image Processing Toolbox have support for GPU processing in R2013b. This means you can accelerate existing MATLAB scripts and functions with minimal changes. To find the list of functions that are supported for GPU  processing in the Image Processing Toolbox, you can visit this<\/a> page. Some of the basic image processing algorithms like image filtering, morphology and edge detection have GPU support and this list is going to grow in the coming releases.<\/p>
Let's look at a small example to set the ball rolling. Inspired by Brett Schoelson's guest post<\/a> a few months back about Photoshop-like effects in MATLAB, I thought I might do one of my own. I call it the canvas effect<\/i> . The canvas effect gives an image the feel of a canvas painting. I had created this little function that does it.<\/p>
type canvasEffect<\/span>\r\n<\/pre>
\r\nfunction out = canvasEffect(im)\r\n\r\n% Filter the image with a Gaussian kernel.\r\nh = fspecial('gaussian');\r\nimf = imfilter(im,h);\r\n\r\n% Increase image contrast for each color channel.\r\nima = cat( 3, imadjust(imf(:,:,1)), imadjust(imf(:,:,2)), imadjust(im(:,:,3)) );\r\n\r\n% Perform a morphological closing on the image with a 11x11 structuring\r\n% element.\r\nse = strel('disk',9);\r\nout = imopen(ima,se);\r\n<\/pre>
It's fairly straight-forward. I first smooth the image with a Gaussian kernel to round off some edges. Then to give the effect of more vivid colors, I increase the contrast for each color channel and finally a morphological opening gives it the canvas painting look. Ofcourse, you could add more bells and whistles by providing additional inputs for the filter kernel size and structuring element, but I wanted to keep it simple.<\/p>
The script below reads an aerial image and gives it that canvas painting effect.<\/p>
type canvasAerialCPU<\/span>\r\n<\/pre>
\r\n% Read the image.\r\nim = imread('concordaerial.png');\r\n\r\n% Produce canvas effect.\r\ncanvas = canvasEffect(im);\r\n\r\n%Display the canvas-ed image.\r\nfigure; imshow(canvas);\r\n<\/pre>
All the processing in the script above was done on the CPU. To move the computation to the GPU, I need to transfer the image from the CPU to the GPU using the gpuArray constructor. So the new script would like this:<\/p>
type canvasAerialGPU<\/span>\r\n\r\nrun canvasAerialGPU<\/span>\r\n<\/pre>
\r\n% Read the image.\r\nim = imread('concordaerial.png');\r\n\r\n% Transfer data to the GPU.\r\nimGPU = gpuArray(im);\r\n\r\n% Produce canvas effect.\r\ncanvasGPU = canvasEffect(imGPU);\r\n\r\n% Gather data back from the GPU.\r\ncanvas = gather(canvasGPU);\r\n\r\n%Display the canvas-ed image.\r\nfigure; imshow(canvas);\r\n<\/pre>\"\" 
Wasn't that easy! All I had to do was convert the image to a gpuArray and gather data back after all the computation was done. The function canvasEffect did not have to change at all. This was because all functions used in canvasEffect were supported for GPU computing.<\/p>
Let's see how much of a win this is in terms of performance. For a few years now I've been using the timeit function that Steve put on the File Exchange. From R2013b, the timeit<\/a> function is part of MATLAB.<\/p>
cpuTime = timeit(@()canvasEffect(im), 1)\r\n<\/pre>
\r\ncpuTime =\r\n\r\n    3.1311\r\n\r\n<\/pre>
This function however can only be used to benchmark computations undertaken by the CPU. For the GPU, a special benchmarking function gputimeit<\/a> has been provided. This function ensures that all computations have completed on the GPU before recording the finish time.<\/p>
gpuTime = gputimeit(@()canvasEffect(imGPU), 1)\r\n<\/pre>
\r\ngpuTime =\r\n\r\n    0.2130\r\n\r\n<\/pre>
So with these small changes, I was able to get a considerable speed-up. Imagine having to do this on an entire data set of images. Working with the GPU would save a lot of processing time.<\/p>
speedup = cpuTime\/gpuTime\r\n<\/pre>
\r\nspeedup =\r\n\r\n   14.6990\r\n\r\n<\/pre>
This is not the complete picture though. I have not accounted for the time it takes to transfer data from the CPU to the GPU and back. This may or may not be significant, depending on how long the computations themselves take. As a rule of thumb, minimize data transfers to and from the device.<\/p>
transferTimeToGPU = gputimeit(@()gpuArray(im), 1)\r\ntransferTimeToCPU = gputimeit(@()gather(canvasGPU), 1)\r\n\r\ngpuTime = transferTimeToGPU + gpuTime + transferTimeToCPU;\r\n\r\nspeedup = cpuTime\/gpuTime\r\n<\/pre>
\r\ntransferTimeToGPU =\r\n\r\n    0.0037\r\n\r\n\r\ntransferTimeToCPU =\r\n\r\n    0.0074\r\n\r\n\r\nspeedup =\r\n\r\n   13.9753\r\n\r\n<\/pre>
I'm going to end with some pointers about the performance of GPU processing.<\/p>
  1. We've seen in the simple example above that you can get a significant speed-up using the supported functions. However, this speed-up is highly dependent on your hardware. If you have a very capable CPU with multiple cores and a not-so-good GPU, the speed-up can appear to be poor because functions like imfilter and imopen are multi-threaded on the CPU. Similarly, if you have a reasonable GPU on a not-so-capable CPU, you're speed-up can make you're GPU execution look faster than it is.<\/li>
  2. The speed-up achieved is dependent on image size. At smaller image sizes, the overhead of parsing input arguments and moving data to and from the GPU contribute to lower speed-ups. Here's an example that demonstrates this.<\/li><\/ol><\/div>
    % Define image sizes over which to measure performance.<\/span>\r\nsizes = [100 500 2000 4000];\r\n\r\n% Preallocate timing arrays.<\/span>\r\n[cpuTime,gpuTime,transferTimeToGPU,transferTimeToCPU] = deal(zeros('like'<\/span>,sizes));\r\n\r\nfor<\/span> n = 1 : numel(sizes)\r\n    size = sizes(n);\r\n\r\n    % Resize image to size x size.<\/span>\r\n    im_scaled = imresize(im,[size size]);\r\n\r\n    % Transfer resized image to GPU.<\/span>\r\n    imGPU_scaled = gpuArray(im_scaled);\r\n\r\n    % Process image on GPU.<\/span>\r\n    canvasGPU_scaled = canvasEffect(imGPU_scaled);\r\n\r\n    % Time CPU execution.<\/span>\r\n    cpuTime(n)           = timeit(@()canvasEffect(im_scaled), 1);\r\n\r\n    % Time GPU execution.<\/span>\r\n    transferTimeToGPU(n) = gputimeit(@()gpuArray(im_scaled)       , 1);\r\n    gpuTime(n)           = gputimeit(@()canvasEffect(imGPU_scaled), 1);\r\n    transferTimeToCPU(n) = gputimeit(@()gather(canvasGPU_scaled)  , 1);\r\nend<\/span>\r\n\r\ngpuTotalTime = transferTimeToGPU+gpuTime+transferTimeToCPU;\r\n% Plot CPU vs GPU execution<\/span>\r\nfigure;\r\nplot(sizes, cpuTime, 'rx--'<\/span>,...<\/span>\r\n     sizes, gpuTotalTime,'bx--'<\/span>,...<\/span>\r\n     'LineWidth'<\/span>,2);\r\nlegend('cpu time'<\/span>,'gpu time'<\/span>);\r\nxlabel('image size [n x n]'<\/span>);\r\nylabel('execution time'<\/span>);\r\ntitle('cpu time vs gpu time'<\/span>);\r\n\r\nfigure;\r\nplot(sizes,cpuTime.\/gpuTotalTime,'LineWidth'<\/span>,2);\r\nxlabel('image size [n x n]'<\/span>);\r\nylabel('speed up'<\/span>);\r\ntitle('Speed up'<\/span>);\r\n<\/pre>\"\" \"\" 
    I hope this got you as excited about image processing with GPU's as it did me!<\/p>