{"id":8391,"date":"2017-02-10T09:00:27","date_gmt":"2017-02-10T14:00:27","guid":{"rendered":"https:\/\/blogs.mathworks.com\/pick\/?p=8391"},"modified":"2017-02-10T09:52:52","modified_gmt":"2017-02-10T14:52:52","slug":"semaphore-posix-and%e2%80%8b-windows","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/pick\/2017\/02\/10\/semaphore-posix-and%e2%80%8b-windows\/","title":{"rendered":"Semaphore POSIX and\u200b Windows"},"content":{"rendered":"
\n <\/p>\n

Sean<\/a>‘s pick this week is Semaphore POSIX and Windows<\/a> by Andrew Smart<\/a>.\n <\/p>\n

<\/introduction><\/p>\n

Background<\/a><\/h3>\n

I’ve been working with someone to help them parallelize multiple simulations of an external program. The external program can be driven by its COM API on Windows using actxserver<\/tt><\/a>. This is an ideal candidate for parallel computing because the simulation takes minutes to hours to run and needs to be run for different configurations where there are no dependencies between runs.\n <\/p>\n

My first attempt at this was to use a parfor<\/tt><\/a> loop, looping over configurations in parallel. It worked the first time(!), but broke the second time and continued to be unreliable. The problem was that I cannot open the actxserver<\/tt> at the same time from multiple parallel workers because a race condition<\/a> is encountered.\n <\/p>\n

So the challenge: How to enforce that each parallel worker opens the actxserver<\/tt> at a different time than the others, ideally immediately after the other has finished.\n <\/p>\n

The way to do this is with a semaphore<\/a>, which MATLAB does not have built in. A semaphore is used to control access to a shared resource. I vaguely remembered having seen this on the File Exchange before and checked my Pick of the Week candidates list and sure enough, there it was, waiting for a use-case!\n <\/p>\n

Using The Semaphore<\/a><\/h3>\n

NOTE<\/b> To use the semaphore, you need to compile using the following command:<\/p>\n

mex -O<\/span> -v<\/span> semaphore.c<\/span><\/pre>\n<\/p>\n
I will first create the semaphore from the client MATLAB.  The parallel workers will grab it before opening the actxserver<\/tt> and release it after.  The first worker to execute will grab the semaphore, making the other workers enter a First-In-First-Out queue to repeat the process.  After a few seconds of initialization time, all of the workers will be running the lengthy simulation in parallel.\n   <\/p>\n
Open parallel computing pool.  This pool has two workers because I have a dual core laptop.<\/p>\n
gcp<\/pre>\n
ans = \r\n Pool with properties: \r\n\r\n            Connected: true\r\n           NumWorkers: 2\r\n              Cluster: local\r\n        AttachedFiles: {}\r\n          IdleTimeout: 90 minute(s) (85 minutes remaining)\r\n          SpmdEnabled: true\r\n<\/pre>\n
Create semaphore with key 1 and value 1.  The value here matters, use 1.<\/p>\n
semkey = 1;\r\nsemaphore('create'<\/span>,semkey,1)<\/pre>\n
Run the parfor<\/tt>-loop\n   <\/p>\n
parfor<\/span> ii = 1:4\r\n    try<\/span>\r\n        semaphore('wait'<\/span>,semkey)\r\n        disp(string('Iteration '<\/span>) + ii + ' Started: '<\/span> + string(datetime('now'<\/span>)))\r\n        pause(2) % Proxy for actxserver()<\/span>\r\n        semaphore('post'<\/span>,semkey)\r\n        disp(string('Iteration '<\/span>) + ii + ' Finished: '<\/span> + string(datetime('now'<\/span>)))\r\n        % Run the simulation here<\/span>\r\n\r\n    catch<\/span>\r\n        % In case something goes wrong, release the semaphore<\/span>\r\n        semaphore('post'<\/span>,semkey)\r\n\r\n    end<\/span>\r\nend<\/span><\/pre>\n
Iteration 3 Started: 08-Feb-2017 11:09:34\r\nIteration 2 Started: 08-Feb-2017 11:09:32\r\nIteration 2 Finished: 08-Feb-2017 11:09:34\r\nIteration 1 Started: 08-Feb-2017 11:09:37\r\nIteration 3 Finished: 08-Feb-2017 11:09:37\r\nIteration 1 Finished: 08-Feb-2017 11:09:39\r\nIteration 4 Started: 08-Feb-2017 11:09:39\r\nIteration 4 Finished: 08-Feb-2017 11:09:41\r\n<\/pre>\n
If you look at the times, you will see that iteration 2 started at 32 seconds, and finished two seconds later (parfor<\/tt> iterations do not run in sorted order).  Iteration 3 starts immediately after 2 and when it finishes iteration 1 begins.  Iteration 4 starts after iteration 1 completes.\n   <\/p>\n
Here’s an annotated image:<\/p>\n
 <\/p>\n
Clean up after.<\/p>\n
semaphore('destroy'<\/span>,semkey)<\/pre>\n
I was able to get this working in just a few minutes but did learn a couple lessons the hard way.  A few things to note:<\/p>\n
\n
    \n
  • Use try\/catch<\/tt> around the operations you’re doing.  If the process fails, you want to release the semaphore to unblock the other workers.  This can cause a hang if an error occurs because the loop will never complete as other workers are held.  If you reach this state, force kill the parallel pool from the task manager.\n         <\/li>\n
  • Use Key=1<\/tt> and Value=1<\/tt> incrementing Key<\/i> if you have multiple semaphores.  My only advice to Andrew is to make this a little more clear in the  help.\n         <\/li>\n<\/ul><\/div>\n
    Comments<\/a><\/h3>\n
    Give it a try and let us know what you think here<\/a> or leave a comment<\/a> for Andrew.\n   <\/p>\n