{"id":2886,"date":"2024-11-18T11:05:27","date_gmt":"2024-11-18T16:05:27","guid":{"rendered":"https:\/\/blogs.mathworks.com\/matlab\/?p=2886"},"modified":"2025-09-23T05:28:21","modified_gmt":"2025-09-23T09:28:21","slug":"20-years-of-supercomputing-with-matlab","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/matlab\/2024\/11\/18\/20-years-of-supercomputing-with-matlab\/","title":{"rendered":"20 years of supercomputing with MATLAB"},"content":{"rendered":"
While prepping for the <\/span>SC24 supercomputing conference<\/span><\/a> with the rest of the MathWorks crew I idly wondered when <\/span>Parallel Computing Toolbox <\/span><\/a>was first released. <\/span>Jos Martin<\/span><\/a>, one of the original authors of the toolbox, glanced at his watch before saying \"oh wow! Actually 20 years yesterday! <\/span>The first version of Distributed Computing Toolbox, which later evolved into the products we now call Parallel Computing Toolbox and MATLAB Parallel Server, was released on November 7th 2004<\/span>\". <\/span><\/div>
\"Someone should tell the story!\" I insisted and, as it turns out, volunteered. So, Jos sat down and started to tell me how things were back in the day. <\/span><\/div>

1995: Cleve Moler writes \"Why there isn't a parallel MATLAB\"<\/span><\/h2>
All good stories of how a thing was done starts with the story of why the thing shouldn't be done and this one is no different. Back in 1995, a parallel version of MATLAB didn't make much sense and Cleve was moved to write about exactly why this was the case in an article he <\/span>tells us is the most widely cited piece<\/span><\/a> he's ever written: <\/span>Why there isn't a parallel MATLAB<\/span><\/a>.<\/span><\/div>
That article considered what we now call <\/span>distributed arrays<\/span><\/a>: matrices\/arrays that are split across multiple independent computers. 1995 computer technology just wasn't good enough to make a performant parallel MATLAB that would work in any way that was recognizable as MATLAB. The results were simply too slow. In addition, barely anyone actually had a decent parallel computer. It didn't make good business sense to put too much effort into this.<\/span><\/div>

2004: Distributed Computing Toolbox debuts in SC04, Pitsburgh.<\/span><\/h2>
Things were rather different in 2004. MathWorks had customers that were demanding a parallel MATLAB and so a team formed to build one. This is where Jos' story began:<\/span><\/div>
Jos: We released it on November 7th 2004 and it was pretty basic!<\/span><\/div>
Me: I guess <\/span>parfor<\/span> and not much else right?<\/span><\/div>
Jos: <\/span>parfor<\/span>?! Oh you sweet summer child! <\/span>parfor<\/span> wouldn't be out until R2007b. Version 1.0 supported independent jobs using the <\/span>MATLAB Job scheduler<\/span><\/a> and nothing else. We didn't have <\/span>spmd<\/span><\/a> and we didn't have communicating jobs or MPI. We didn't support other schedulers. We didn't have much but it did provide the first building blocks for the tools that allowed people to write parallel algorithms in MATLAB without MPI, and that was what was needed at the time.<\/span><\/div>
Me: OK, so you had the <\/span>batch<\/span><\/a> command?<\/span><\/div>
Jos: No, that came later. We had the precursor to <\/span>createJob<\/span><\/a> and <\/span>createTask<\/span><\/a>. What you could do was say \"Run this over there, and run that over at this other place\". You could do this with multiple clusters, like you can do with <\/span>parcluster<\/span><\/a> today. So, a customer could have a local MATLAB program that would send tasks to other machines. Those machines could be running Windows or Linux and you could use all of them at the same time.<\/span><\/div>
It was basic but it worked and even then it was very useful to some of our users. This is what the demo looked like at the MathWorks Booth at SC04.<\/span><\/div>
\"The<\/img><\/div>

2005: Version 2.0 and Bill Gates at the SC05 conference<\/span><\/h2>
Version 2.0 had support for communicating jobs via MATLAB Job Scheduler and we also started supporting 3rd party schedulers, but only for non-communicating jobs back then. One of the 3rd party schedulers we supported was CCS, which later became HPC Server on Windows. <\/span><\/div>
In the picture below you have Bill Gates on the SC stage in Seattle running MATLAB on a Windows cluster. A short while later, Kyril Faenov, director of the High Performance Computing product unit in the Windows Server Group, came on stage to run a demo. The live demo distributed an algorithm that analyzed proteomics data on a local multinode Windows cluster and then transferred the job to a 64-node Linux cluster that was based in Washington.<\/span><\/div>
\"\"<\/img><\/div>
Jos was in the front row, sat next to Cleve, watching rather nervously. He'd been out there for 3 weeks in advance making sure that everything worked OK. I can only imagine what was going through his mind at that moment. Think about it: code you had written was being used in a live demo at <\/span>the <\/span>largest event in your industry, as part of a keynote by the CEO of Microsoft, while the co-founder of your company was sat next to you. If it were me, the stress of it all would have resulted in a heart attack moments before the demo started. <\/span><\/div>
Kyril pressed the go button, the CPU core load shot up and results start coming back. Jos breathed out a sigh of relief, the live demo had worked and it all looked pretty cool! <\/span><\/div>
Jos went backstage after the keynote to thank the team who were babysitting the clusters.<\/span><\/div>
Jos: <\/span>\"Guys! We made it. It was awesome! Thank you.\"<\/span><\/div>
Them: <\/span>\"Just! 20 seconds after the demo finished, we lost the InfiniBand connection and everything crashed\"<\/span><\/div>
Back then, the MathWorks booth at SC looked like the picture below. This was in the days when we still had a rack of real machines on-premise to run the demos. <\/span>Cleve tells us<\/span><\/a> that the rack itself was bought from Home Depot!<\/span><\/div>
\"The<\/img><\/div>

2007: Local workers, parfor and Cleve's follow up article<\/span><\/h2>
For the first few years, everything in parallel MATLAB was <\/span>only<\/span> HPC in the sense that you ran parallel workers on other machines. There was no concept of having a local pool so you couldn't run multiple workers on the same machine. <\/span><\/div>
This was long before my time at MathWorks but looking at the history, it makes sense. The first dual core Intel and AMD chips were released in 2005 so it would be a while before your average MATLAB user felt the pain of not being able to run multiple workers on the same machine.<\/span><\/div>
Local workers really changed the game though. You could prototype your parallel code on your local dual core machine and then it would run <\/span>without modification<\/span> on a HPC cluster, you could just scale up the number of workers. <\/span><\/div>
2007 was the year that <\/span>parfor<\/span><\/a> became a thing and this really changed the game. Usually the first part of Parallel Computing Toolbox that most users learn, <\/span>parfor<\/span> is the easy way to parallelize loops in MATLAB. It has been developed a lot since that first version and has all sorts of tricks up its sleeve that aren't known by many people. I'll do a deep dive on it at some point next year.<\/span><\/div>
Cleve also wrote a follow up to his 1995 article. This one was called <\/span>Parallel MATLAB: Multiple processors and multiple cores<\/span><\/a> that gave an overview of parallel computing in MATLAB at that time.<\/span><\/div>

2008: MathWorks jointly won the SC08 HPC Challenge class 2 award<\/span><\/h2>
The HPC Challenge is a benchmark that has obvious winners - the best result. There is also a class 2 award that is a little more subjective as it awards the most \"elegant\" implementation of four or more of the HPC Challenge benchmarks. This award was weighted 50% on performance and 50% on code elegance, clarity, and size. <\/span><\/div>
In SC08, the MATLAB team jointly won the class 2 award. We shared the prize with Chapel and X10 who also turned in great submissions. <\/span>One of the examples in today's MATLAB documentation<\/span><\/a> is based on this work which allows you to run the challenge using MATLAB today.<\/span><\/div>

2010: GPU computing in MATLAB debuted at SC10 and NVIDIA GTC<\/span><\/h2>
These days, <\/span>MATLAB has over 1,000 functions that support execution on NVIDIA GPUs<\/span><\/a> but the first version, released as part of R2010b, was rather more limited. Two glaring omissions were the <\/span>subsref<\/span> and <\/span>subsasgn<\/span> functions which meant that indexing of <\/span>gpuArray<\/span>'s was not supported. Imagine trying to do anything useful in MATLAB without ever indexing anything!<\/span><\/div>
Even then, however, <\/span>gpuArray<\/span> worked with 123 functions and it was a big enough release that Jensen Huang highlighted it in the 2010 NVIDIA conference.<\/span><\/div>
\"\"<\/img><\/div>
GPU computing was also the highlight of MathWorks' attendance at SC10. The booth is starting to look a little more like it does today.<\/span><\/div>
\"\"<\/img><\/div>

2013: parfeval gets added to the parallel language<\/span><\/h2>
parfeval<\/span><\/a> is a function in Parallel Computing Toolbox that doesn't get enough coverage. It allows you to request asynchronous execution of a function on a worker, This provides the programmer with the means to write more expressive code than is possible with <\/span>parfor<\/span> in a way that's easier than low-level functions such as <\/span>spmdSend<\/span>, <\/span>spmdReceive<\/span> and friends. <\/span><\/div>
In future years, additional support functions, such as <\/span>afterEach<\/span><\/a> and <\/span>afterAll<\/span><\/a>, were added to the language that assisted with workflows based on <\/span>parfeval.<\/span> <\/span><\/div>
If the only parallel function you know in MATLAB is <\/span>parfor<\/span>, I suggest that you take a closer look at how <\/span>parfeval<\/span> might be able to help you.<\/span><\/div>

2015: Cleve's trip report of SC15<\/span><\/h2>
Cleve wrote a <\/span>great report of his time at SC15 on his blog<\/span><\/a>. One of the demos at the MathWorks booth was of a Raspberry Pi, running code generated by MATLAB, doing live edge detection of a video feed,<\/span><\/div>