# Running Scripts on a Cluster Using the Batch Command in Parallel Computing Toolbox

Posted by Loren Shure,

I'd like to introduce this week's guest blogger Edric Ellis. Edric works for the Parallel Computing development team here at The MathWorks. In this post he will talk about using the batch command in Parallel Computing Toolbox.

The problem we're going to look at is a Monte-Carlo simulation of a financial instrument - the expected returns from playing hands of blackjack in a casino. We'll approach the problem in three phases:

• Running the Blackjack Simulation in MATLAB
• Submitting Batch Jobs to Run Blackjack on the "local" Cluster
• Adding parfor Loops to the Script

### Running the Blackjack Simulation in MATLAB

The calculation of the result of playing a single hand of blackjack was written by Cleve Moler, and is available on the file exchange. During this article, we're going to use a slightly modified version of that program where we've extracted just the calculation of a single hand of blackjack so that we can invoke it in a variety of different ways. There's a discussion of the algorithm in chapter 9 of Cleve's book "Numerical Computing with MATLAB".

Cleve describes the amount won playing a single hand of blackjack as follows:

"[win] zero for a 'push', win $15 for a blackjack, win or lose$10 on a hand that has not been split or doubled, win or lose $20 on hands that have been split or doubled once, and win or lose$30 or $40 on hands that have been doubled after a split. The$30 and \$40 payoffs occur rarely (and may not be allowed at some casinos)."

We can calculate the winnings of playing several individual hands of blackjack simply by invoking blackjack_kernel a few times.

winnings = zeros(1,10);
for ii=1:10
winnings(ii) = blackjack_kernel;
end
winnings
winnings =

15   -10     0     0   -10    10   -10   -10   -10    10



### Running Blackjack as a Script Directly in MATLAB

The script blackjack_script simulates a number of players playing many hands of blackjack, and computes for each player their net profit after those hands. Let's run the script and then plot the results as a histogram showing the distribution of profit and loss among the players.

type blackjack_script

blackjack_script;
hist(results);
title('Profit and loss playing blackjack');
xlabel('Profit'); ylabel('Count');
numHands   = 500;
numPlayers = 50;
results    = zeros(1, numPlayers);
tic
for h = 1:numHands
thisHand = zeros(1, numPlayers);
for p = 1:numPlayers
thisHand(p) = blackjack_kernel;
end
results = results + thisHand;
end
t = toc;
fprintf('Time to simulate %d players playing %d hands of blackjack: %.3g seconds\n', ...
numPlayers, numHands, t);

Time to simulate 50 players playing 500 hands of blackjack: 8.04 seconds


### Submitting Batch Jobs to Run Blackjack on the "local" Cluster

Now that our script works correctly, we can submit the script for execution on a cluster. In this case, we're going to use the "local" cluster since it's always available, but you might submit the script for execution on a remote cluster if you have one available. When you do submit to a remote cluster, you're free to close down your MATLAB session and collect the results later. If you're using the local cluster, you can carry on using your MATLAB session while the batch job runs in the background; also, you can have several batch jobs running simultaneously.

We invoke the batch command with the name of the script we wish to run, and here we're also specifying which 'Profile' we wish to use. Profiles identify connections you have made to different clusters that might be available, the 'local' profile is always available. Other cluster types require an MDCS installation. If you only have a single Profile, you don't need to specify the Profile argument. The return from the batch command is a job object. The job object allows us to track the progress of execution on the cluster, and when it has finished, we can access the results.

As well as using the batch command directly from the MATLAB command line, you can right-click on a script file in MATLAB's Current Folder browser, and select "Run Script as Batch Job". This uses your default cluster profile.

In this example, the first thing that we do with the job is call wait so that we only proceed when the job has finished executing. Alternatively, you can carry on working in MATLAB and check the state of the job by looking at job.State, loading the results when the state is 'finished'.

job = batch('blackjack_script', 'Profile', 'local');

% Wait for the job to complete execution.
wait(job);

% Load the results of executing the job into the MATLAB workspace

% And display the diary output from the job
disp('Diary output from the batch job:')
diary(job);

% We can plot the results as we did previously.
hist(results);
title('Profit and loss playing blackjack');
xlabel('Profit'); ylabel('Count');
Diary output from the batch job:
Time to simulate 50 players playing 500 hands of blackjack: 7.93 seconds



### Adding parfor Loops to the Script

Running a batch job on the local machine doesn't make things go any quicker, although running a job on a remote cluster might give you access to a more powerful machine. Even on the local machine, we can still use parfor to gain a speedup. Each iteration of the outer for loop in blackjack_script is independent, and the calculation of results is a "reduction" or summary operation that parfor can understand. So, we can recast the script into a variant using parfor simply by changing the outer loop:

type blackjack_script_parfor
numHands   = 500;
numPlayers = 50;
results    = zeros(1, numPlayers);
tic
parfor h = 1:numHands
thisHand = zeros(1, numPlayers);
for p = 1:numPlayers
thisHand(p) = blackjack_kernel;
end
results = results + thisHand;
end
t = toc;
fprintf('Time to simulate %d players playing %d hands of blackjack: %.3g seconds\n', ...
numPlayers, numHands, t);



### Running the parfor script with an open MATLAB pool

We can check that the parfor loop is working correctly by opening an interactive matlabpool session. An interactive matlabpool session launches workers based on the Profile you specify, and makes them available to your MATLAB session for running the body of parfor loops. Normally, it's a good idea to leave matlabpool open while you're working, but we're going to close it just after running the script to free up workers so that we can run a batch job.

% Open matlabpool using the 'local' profile
matlabpool open local

% Run our script
blackjack_script_parfor

% Close the pool again to free up workers for when we run the batch job
matlabpool close
Starting matlabpool using the 'local' profile ... connected to 6 labs.
Time to simulate 50 players playing 500 hands of blackjack: 1.87 seconds
Sending a stop signal to all the labs ... stopped.


### Running a Batch Job Containing a parfor Loop

To run a batch job where a parfor loop is used, we need to specify an additional argument to the batch command - 'Matlabpool'. The value of the option specifies how many additional workers to use when running the script. The machine I'm using has 6 cores, so I can run 1 worker to control the script execution, and 5 additional workers to act as a matlabpool. These additional workers operate on the body of the parfor loop in parallel to calculate the result more quickly.

job = batch('blackjack_script_parfor', ... % Run the script containing parfor
'Profile', 'local', ...        % using the 'local' profile
'Matlabpool', 5);              % with a Matlabpool of size 5

% Wait for the job to complete execution.
wait(job);

% Load the results of executing the job into the MATLAB workspace

% And display the diary output from the job
disp('Diary output from the batch job using parfor:')
diary(job);

% We can plot the results as we did previously.
hist(results);
title('Profit and loss playing blackjack');
xlabel('Profit'); ylabel('Count');
Diary output from the batch job using parfor:
Time to simulate 50 players playing 500 hands of blackjack: 1.92 seconds



### Conclusions

We have seen how using the batch command, we can send a script to be executed on a cluster. We've used the 'local' cluster here, which is a part of Parallel Computing Toolbox, but we could have just as easily used a remote cluster if a MATLAB Distributed Computing Server installation was available to us. Using a remote cluster allows even more compute resources to be used during the execution of a job. Some for loops (where each iteration can be executed independently of the others) lend themselves to parallel execution using parfor, and we saw how we can run a batch job where our script contained a parfor loop.

There are more details about getting your code running on a cluster in the documentation. Do you have any other questions about running your code on a cluster? Let us know here.

Get the MATLAB code

Published with MATLAB® 7.14