It should be possible to debug local workers simply locating the MATLAB process in task manager, and right-clicking on the worker process and selecting the “Debug” menu item. This will bring up the Visual Studio “just in time” debugger dialog. If you build your mex file with “-g”, you should then be able to set breakpoints within your mex function source code. (I just verified that I could do this successfully, so let me know if you have problems here)

Having said that, I would not expect mexGetVariablePtr() to work as expected, as that violates workspace transparency - in other words, it attempts to read stuff from another workspace. In the same way, “evalin(’caller’, …)” is disallowed.

When executing the body of a parfor loop, we only transmit from the client to the workers those variables that we can see (by analysing the code) are read from within the loop. This allows us to avoid sending the whole contents of the workspace. So, to work effectively with parfor, the variables used inside the body of the loop must be passed as arguments to your mex file.

One possible approach to avoiding sending large constant data repeatedly is to have the workers cache the data inside a persistent function workspace.

Cheers,

Edric.

(i) how do I debug code executing within the parfor loop, preferably both in Matlab and in C (I use ms visual studio 2005)? There does not seem to be a discussion of this in the relevant literature, but maybe I missed it.

(ii) Is it possible that the problem stems from the use of low level gateway routines? e.g. mexGetVariablePtr() to read Matlab memory locations as I’m not certain how the mex code knows to look for worker copies as opposed to the originally stored copy (and I’m not certain this should make a difference without debugging). I avoid using the mexPut… routines but have too many inputs (>50) for the fetching of data, and prefer to pass pointers to large constant arrays rather than copying data to save time and memory.

Cheers,
Steve

Thank you for your swift reply. And I surely don’t like the trouble ahead… certainly when it come to my research :)

However, I’ve followed your instructions, and benchmarked an svd(A, ‘econ’), with A = rand(1000);

The single core (directly in command window) takes 29 secs. The “non-interactive pmode” approach (my example) takes 13 secs. The “Parallel jobs” approach (your example) takes 32 secs (probably around 3 secs startup time).

Maybe I haven’t distributed my data in the correct manner, so I’ve included my test_par_svd function:
—
function u = test_par_svd (A)
A_dist = distribute(A);
[u_dist, s_dist, v_dist] = svd(A_dist, ‘econ’);
u = gather(u_dist);
s = gather(s_dist);
v = gather(v_dist);
clear u_dist; clear s_dist; clear v_dist;
clear A_dist;
end
—

The results “o” also contains 4 times the “u” result from the function above, which kind of indicates that the labs cannot distribute the task among themselves, and every lab calculates the svd of the whole matrix in parallel, but not via a distributed parallel LAPACK SVD. Since my example uses less time, it surely is possible on my Intel MKL lib (version 9).

Please also note that I find it difficult to translate the easy example on parallel jobs (matlab docs) towards the use of an svd function, for which I can’t control the calculation (function of MKL lib).

I’m now strugling with getting the “svds” function to work. MKL doesn’t support this, so I’ve reworked my calculation via the “eigs” function (of the A*A’ matrix), which gives the same results. But apparently this eigs function isn’t supported in MKL 9 (it is in MKL 10). Does anybody know how to change the libs to MKL 10? (I’ve tried a couple of ways described in the matlab docs, but I really need the specific changes to blas.spec (or env. vars.) to get this to work I guess)

Best regards, and thanks a lot already for your time and effort,
Paul

pjob = jm.createParallelJob();
t    = job.createTask(@myfunction, numOutputs, {input1, input2}));
pjob.submit;

%% optionally: pjob.waitForState('Finished');
o = pjob.getAllOutputArguments;
celldisp(o);

Using undocumented API is a surefire way to get into massive trouble.

A much easier way is to use the documented “Parallel Job” API. See documentation: http://www.mathworks.com/access/helpdesk/help/toolbox/distcomp/bqur73g.html

Here is a quick recipe for going from interactive to non-interactive way:
1. Once you have tested your algorithm in “pmode”, create a MATLAB function file using the commands/function calls that worked.

1a. An easy way to do this is to enable command history in “pmode” window, select the commands that worked (CTRL+Click for multiple commands on Windows), right click and choose “Create M-File”.

2. Make it a MATLAB function, i.e., add “function” etc. at the top of the file .

3. Create a parallel job (createParallelJob), add a task to it (createTask) and supply the MATLAB function you created as an input to the task creation function.

4. The createParallelJob function returns a job object which you can submit and later retrieve results, check status etc.

4a. Add FileDependencies to this job object if your function uses other functions created by you.

I noticed you are using the ‘local’ configuration. So here is what you can do:

jm = findResource('scheduler', 'configuration', 'local'); 

and then follow the regular steps for a parallel job. For example:

pjob = jm.createParallelJob();
t   = job.createTask(@myfunction, numOutputs, {input1, input2}));
pjob.submit;
%optionally: pjob.waitForState('Finished'); o = pjob.getAllOutputArguments; celldisp(o); 

NR_OF_LABS = 4;

% opening pmode on 4 labs
pmode (’start’, ‘local’, NR_OF_LABS);

% Distributing the array A over the labs
% Copy A to the labs
pmode(’client2lab’, input_variable, strcat(’1:’, int2str(NR_OF_LABS)));
% Distribute on the labs
iRunCmdOnLabs(strcat(’A_dist = distribute(’, input_variable, ‘);’));

% Calculate the svd in a distributed manner
iRunCmdOnLabs(’[u_dist, s_dist, v_dist] = svd(A_dist , ”econ”);’);

% Gather the data back into every lab
iRunCmdOnLabs(’u = gather(u_dist); s = gather(s_dist); v = gather(v_dist);’);

% Clear intermediary results
iRunCmdOnLabs(’clear u_dist; clear s_dist; clear v_dist; clear A_dist’);

% Send results back to the client
pmode lab2client u 1;
pmode lab2client s 1;
pmode lab2client v 1;

% Clean up the parallel labs
pmode cleanup;

end

function labs = iGetLabs()
%iGetLabs Can be called on the client to get the labs object.
if ~iIsOnClient()
error(’Cannot execute par_svd from labs’);
return;
end
try
session = com.mathworks.toolbox.distcomp.pmode.SessionFactory.getCurrentSession;
labs = session.getLabs();
catch
error(’distcomp:pmode:NotRunning, cannot execute par_svd’);
end
end

function iRunCmdOnLabs(cmd)
%iRunCmdOnLabs Send a command to the labs
session = com.mathworks.toolbox.distcomp.pmode.SessionFactory.getCurrentSession;
if isempty(session)
error(’distcomp:pmode:NotRunning’, …
‘Cannot execute par_svd when pmode is not running.’);
end
% Error messages will only be displayed in the main MATLAB command window, and
% the command will only be executed in the MATLAB client when it is idle.
fprintf(’Sending command %s to the MATLAB labs for evaluation.’, cmd);
labs = iGetLabs();
labs.evalConsoleOutput(cmd);
end

function onclient = iIsOnClient()
onclient = ~system_dependent(’isdmlworker’);
end
—

If anybody know a more elegant solution, please let me know!

Ps: Use the code at own risk. One method (labs.evalConsoleOutput(cmd);) on the labs was undocumented, but seems to work. Have fun, and save time!

Best regards,
Paul

And in the parallel command window:
P>> A_dist = distribute(A);
P>> [u, s, v] = svd(A_dist, ‘econ’);
P>> u_comb = gather(u);

Back in the matlab command window
>> pmode lab2client u_comb;

I obtained a significant speedup (6,8 secs compared to 17 secs). However, I would like to do this non-interactive from an M-file.

Ps: Doing more or less the same with the matlabpool command clearly indicates that no speedup is obtained. So I concluded that matlabpool only works for the parfor loop, is this correct?

Can anybody guide me in the right direction?

Thanks you,
Paul

However, I now want to

Best,

Narfi

a) I substituted parfor for a for loop in a script. A function is called in this loop. Somehow parfor does not look for the function in the same path as the for loop and was calling another version of that function in another directory. How is parfor looking for a function differently than for?

b) I have a struct that I declare outside of the for loop but one field is assigned within the loop. example
s.f1=1;
s.f2=2;
for i=1:10
s.f3=i
output(i)=myfunction(s);
end

This works fine in a for loop, but the parfor loop seems to erase the comple s struct instead of just the field f3. Is this normal?

c) I can’t seem to pass in a cell array of strings which is not assigned inside the loop, it seems to get erased as well. Example
Mystringarray; %cell array of strings
for n=1:length(Mystringarray)
s.f3=Mystringarray(n);
end

Now when i use parfor, it seems like Mystringarray does not exist after the first iteration of the parfor.

Thanks!!!

Best,

Narfi