MATLAB’s High Performance Computing (HPC) and ‘Big Data’ datatypes
Table of Contents
I'm currently preparing for SC22, my first supercomputing conference as a MathWorker. While making plans with old friends from other High Performance Computing organisations who don't know MATLAB so well, I'm often asked 'HPC in MATLAB....What else is there besides parfor?'
parfor is great, allowing users to go from a serial loop to a parallel one with the addition of only 3 letters, but it's just the beginning of the possibilities for parallel and scalable programming in MATLAB.
Today, I wanted to give a brief survey of what I think of as MATLAB's HPC data-types which allow users to make use of GPUs, distributed memory and work with out-of-core data.
gpuArrays - GPU programming made easy
GPUs, or Graphics Processing Units, can drastically accelerate certain types of computation. MATLAB currently only supports NVIDIA GPUs which are traditionally programmed using a language called CUDA. You can code in CUDA if you want to, and interface to MATLAB using CUDA mex functions....but there's an easier way.
Say you have an array, X
X = [1,2,3];
transfer this to the GPU with
G = gpuArray(X);
Any calculation you subsequently do on the gpuarray G will be automatically performed on the GPU.
Gsq = G.^2; % Performed on the GPU
Gsin = sin(G); % Performed on the GPU
As with the other 'HPC datatypes', you use gather to transfer a result back to a 'normal' MATLAB matrix
Xsq = gather(Gsq);
Xsin = gather(Gsin);
dlarray - Specialised objects for Deep Learning training
dlarray is an array type for doing deep learning. It supports Automatic Differentiation and can contain 'normal' arrays that get evaluated by the CPU as well as gpuArrays. How to use dlarrays for deep learning is beyond the scope of this article so I'll just show to create them and link to the documentation.
X = randn(3,5);
dlX = dlarray(X) % Anything you do with dlX will be evaluated using the CPU
and if we want our dlarray to be created on the GPU we do
gX = randn(3,5,'gpuArray');
dlgX = dlarray(gX) % Anything you do with dlgX will be evaluated using the GPU
- Custom Training and Calculations using Automatic Differentiation
- List of functions with dlarray support
Tall arrays - For when you've got more rows than memory
Tall arrays are useful for when you have tabular data with millions, or even billions of rows. So many rows, in fact, that you could never load them all into memory at once.
Imagine that all of your data is stored in .csv files in a folder called mydata. We start off by creating a datastore that points to all of these files.
ds = datastore("mydata\*.csv");
Note that we have not loaded all of the data into memory, we have merely created an object, ds, that represents and points to the data. We can see its type with
class(ds)
To operate on this datastore we now create a tall array from it. Since I have Parallel Computing Toolbox installed, I'll also start a local pool so all tall array operations are performed in parallel. Without a Parallel Computing Toolbox install everything would be performed serially.
delete(gcp("nocreate")) % Ensure that there is no exisiting parallel pool
parpool("processes");
tt = tall(ds)
This shows a preview of the data. Note that the height of the tall array is unknown (MATLAB just tells us that there are 'M' rows) because we have not yet parsed through all of the files. We can operate on this tall array as if it were any other table in MATLAB.
Let's get the mean of the BigThings column
BigThingsMean = mean(tt.BigThings)
This demonstrates another difference between tall arrays and normal, in-memory MATLAB arrays - Deffered evaluation. Everything remains unevaluated until you explicity request that the calculation is performed. When you make multiple operations on tall arrays, this behaviour allows MATLAB to minimise the number of passes through the data when asked to evaluate the final results.
We request evaluation using the gather command. Since I have Parallel Computing Toolbox, it is automatically done in parallel using the local process pool.
gather(BigThingsMean)
Tall arrays work by only ever having a resonable-sized chunk of the data in memory at once and can also make use of parallel infrastructure such as parallel workers on your local computer, traditional HPC clusters or even Spark clusters.
- Logistic regression with Tall Arrays
- Bayesian Optimization with Tall Arrays
- Statistics and Machine Learning using Tall Arrays
Tall arrays are available in base MATLAB but can be accelerated and scaled using Parallel Computing Toolbox and scaled even further with MATLAB Parallel Server.
Distributed arrays - Spread your arrays across multiple nodes of a HPC cluster
If your matrix is too large to fit onto one node of a HPC cluster, use a distributed array to spread it across multiple nodes.
Just as you can use tall(foo) to create tall arrays, you can use distributed(foo) to create distributed arrays. With a distributed array, the result is in memory but that memory is shared across multiple workers. Ensure that those workers are on multiple nodes of a HPC cluster and fully distributed, parallel computing is yours.
Our first example creates a distributed array A from a normal array X which isn't particulalry useful since if the array fits into memory of a single machine, there isn't much point in using distributed arrays. We'll see more sophisticated, and useful, methods of creating distributed arrays later.
% Created distributed array from normal array X
X = rand(4);
A = distributed(X);
You can work with this variable A as one single entity, without having to worry about its distributed nature. The code for computing a singular value decomposition, for example, is identical to how we'd do it for a normal array.
distres = svd(A); % Computing the svd of the distributed array, A
The result is still a distributed array
class(distres)
When you are ready to bring the results back into client memory, use the gather function, just as you would for a tall array. You have to be careful here though to ensure that we only request results that will fit into our client machine's memory.
% Bring back from cluster
res = gather(distres)
This is now a normal MATLAB array.
class(mysvd)
Here, I've shown the creation of a distributed array from a normal MATLAB array which, when you think about it, is a bit pointless other than for prototyping purposes. If you can fit X into memory, there is nothing to be gained from using distributed arrays instead of normal arrays.
We can also create distributed arrays directlty on the workers using functions such as rand,randn and zeros. This way, the overall matrix never has to fit into memory of the client. For example,
distRandom = rand(5,'distributed');
class(distRandom)
Often, the creation of a distributed array is the most difficult part of a distributed computing workflow! Other methods of creation include using spmd, codistributed arrays (see next section) and datastore. Refer to the documentation for more details.
Over 500 functions are supported for distributed arrays including the vast majority of linear algebra operations (Note for the HPC specialists: we use an implementation of the scalapack library behind the scenes for some of this).
Distributed arrays require Parallel Computing Toolbox as a minimum but you also need MATLAB Parallel Server to make use of multiple node operation.
Codistributed arrays - Advanced moves for distributed array users
distributed arrays are actually a convenient user interface to a lower-level data type called codistributed arrays. The way to think about it is this: A distributed array is an abstraction referring to the entire distributed matrix that you can use without worrying about its distributed narture. The portion of the array that exists on each worker is a codistributed array.
The explicit use of codistributed arrays is a rather advanced move as they are almost always used within spmd blocks which are themselves more advanced parallel constructs than used by many users.
delete(gcp("nocreate")) % Ensure that there is no exisiting parallel pool
parpool("Processes",2); % Create a pool with 2 workers
Now we have a pool, we can run an spmd block
spmd % Every worker runs the contents of the spmd block
format compact
A = zeros(80,1000);
D = codistributed(A)
fprintf("The class of D on this worker is %s",class(D))
end
fprintf("The class of D outside the spmd block is %s",class(D))
The above is a somewhat silly example since each worker creates the entire array A before using the codistributed function to spread it across workers. However, it serves the purpose of showing what's going on behind the scenes. Outside of the spmd block on our client MATLAB, D is a distributed array that refers to the entire matrix. Inside the spmd block, and hence on each worker, D is a codistributed array. We could get at the portion of D stored on each worker using the getLocalPart() function inside an spmd block.
Further discussion of codistributed arrays are beyond the scope of this article but essentially you use them for finer control over your distributed arrays. You could, for example, do something unique with the part stored on each worker or change how a distributed aray is distributed across workers using codistributor
- When to use distributed arrays
- Working with Codistributed arrays (More advanced details)
So there you have it, a quick survey of some datatypes in MATLAB that facilitate High Peformance Computing.
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.