{"id":198,"date":"2008-02-29T11:41:40","date_gmt":"2008-02-29T16:41:40","guid":{"rendered":"https:\/\/blogs.mathworks.com\/steve\/2008\/02\/29\/timing-code-in-matlab\/"},"modified":"2019-10-24T13:57:45","modified_gmt":"2019-10-24T17:57:45","slug":"timing-code-in-matlab","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/steve\/2008\/02\/29\/timing-code-in-matlab\/","title":{"rendered":"Timing code in MATLAB"},"content":{"rendered":"<div class=\"content\">\r\n\r\nWhen we work on improving the speed of image processing functions in MATLAB, we are naturally interested in getting timing\r\nmeasurements. And we know customers are often timing their own code, typically by using the MATLAB functions <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/tic.html\"><tt>tic<\/tt> and <tt>toc<\/tt><\/a>.\r\n\r\nIt turns out to be surprisingly difficult to get reliable, repeatable timing measurements on modern computers. This is true\r\nfor any program, not just MATLAB.\r\n\r\nI've written timing code many different times over the years, using a variety of methods. Along the way I've learned some\r\nimportant tips and pitfalls from people like <a href=\"https:\/\/www.mathworks.com\/company\/aboutus\/founders\/clevemoler.html\">Cleve Moler<\/a> and Bill McKeeman. Late last year I was working on YABP (Yet Another Benchmark Program), and I wondered if I could encapsulate at least some\r\nof those tips into a timer utility function, one that anyone could use without thinking too hard or making too many choices.\r\nThe result was the function <tt>timeit.m<\/tt>, which you can find on the MATLAB Central File Exchange.\r\n\r\n<tt>timeit<\/tt> takes one input argument and returns one output argument:\r\n<pre>   s = timeit(f)<\/pre>\r\n<tt>f<\/tt> is a function handle, and <tt>s<\/tt> is the time, in seconds, required to call that function handle. <tt>timeit<\/tt> calls <tt>f<\/tt> with no input arguments. To use <tt>timeit<\/tt> successfully, it's very helpful to know how to make <a href=\"\">anonymous function handles<\/a>. Here are a couple of examples:\r\n\r\nHow much time does it take to compute <tt>sum(A.' .* B, 1)<\/tt>, where <tt>A<\/tt> is 12000-by-400 and <tt>B<\/tt> is 400-by-12000?\r\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid #c8c8c8;\">A = rand(12000, 400);\r\nB = rand(400, 12000);\r\nf = @() sum(A.' .* B, 1);\r\ntimeit(f)<\/pre>\r\n<pre style=\"font-style: oblique;\">ans =\r\n\r\n    0.1125\r\n\r\n<\/pre>\r\nHow much time does it take to dilate the text.png image with a 25-by-25 all-ones structuring element?\r\n<pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid #c8c8c8;\">bw = imread(<span style=\"color: #a020f0;\">'text.png'<\/span>);\r\nse = strel(ones(25, 25));\r\ng = @() imdilate(bw, se);\r\ntimeit(g)<\/pre>\r\n<pre style=\"font-style: oblique;\">ans =\r\n\r\n    0.0037\r\n\r\n<\/pre>\r\nThe function <tt>timeit<\/tt> is designed to handle three key timing issues automatically:\r\n<div>\r\n<ul>\r\n \t<li>\"Warming up\" the function to be timed, in order to eliminate one-time effects related to file systems, memory systems, M-file\r\nparsing and optimization, etc.<\/li>\r\n \t<li>Adjusting to the granularity of the timer functions <tt>tic<\/tt> and <tt>toc<\/tt>. <tt>timeit<\/tt> does this by calling <tt>f<\/tt> in an \"inner loop,\" with the number of inner loop repetitions set so that each call to <tt>tic<\/tt> \/ <tt>toc<\/tt> measures a time interval of at least about 0.01 seconds. The appropriate number of inner loop repetitions is automatically\r\ndetermined.<\/li>\r\n \t<li>Running an \"outer loop\" that calls and times the inner loop in a set of repeated trials. The median time for the trials is\r\nthen used to compute the time estimate.<\/li>\r\n<\/ul>\r\n<\/div>\r\nAlso, since the calls to <tt>f<\/tt> all occur within a function, the large variability associated with timing code from the command prompt is eliminated.\r\n\r\nThe function is not perfect, and other people might make different choices than I did. For example, you could argue that\r\nI should be using the minimum time instead of the median time for the time trials. I chose median because in my experience\r\nI have found that it takes fewer repetitions to get a repeatable result by using the median. Also, <tt>timeit<\/tt> will call <tt>f<\/tt> at least 13 times, which is possibly more than necessary for some longer-running functions.\r\n\r\nIf you are interested in issues and techniques related to performance measurement, particularly in MATLAB, then I highly recommend\r\nthat you read <a title=\"https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/loadFile.do?objectId=18510&amp;objectType=file (link no longer works)\">Bill's white paper on performance measurement<\/a>.\r\n\r\nIf you have ideas about improving <tt>timeit<\/tt>, I'd be very interested to hear them.\r\n\r\n<script language=\"JavaScript\">\r\n<!--\r\n\r\n    function grabCode_934335e2751d48dab39f28daec0977a6() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='934335e2751d48dab39f28daec0977a6 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 934335e2751d48dab39f28daec0977a6';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        author = 'Steve Eddins';\r\n        copyright = 'Copyright 2008 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<\/p>\r\n<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add author and copyright lines at the bottom if specified.\r\n        if ((author.length > 0) || (copyright.length > 0)) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (author.length > 0) {\r\n                d.writeln('% _' + author + '_');\r\n            }\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\r\n<p>\\n');\r\n      \r\n      d.title = title + ' (MATLAB code)';\r\n      d.close();\r\n      }   \r\n      \r\n-->\r\n<\/script>\r\n<p style=\"text-align: right; font-size: xx-small; font-weight: lighter; font-style: italic; color: gray;\">\r\n<a><span style=\"font-size: x-small; font-style: italic;\">Get\r\nthe MATLAB code\r\n<noscript>(requires JavaScript)<\/noscript><\/span><\/a>\r\n\r\nPublished with MATLAB\u00ae 7.5<\/p>\r\n\r\n<\/div>\r\n<!--\r\n934335e2751d48dab39f28daec0977a6 ##### SOURCE BEGIN #####\r\n%%\r\n% When we work on improving the speed of image processing\r\n% functions in MATLAB, we are naturally interested in getting\r\n% timing measurements.  And we know\r\n% customers are often timing their own code, typically by using\r\n% the MATLAB functions\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/tic.html % |tic| and |toc|>.\r\n%\r\n% It turns out to be surprisingly difficult to get reliable,\r\n% repeatable timing measurements on modern computers.  This is\r\n% true for any program, not just MATLAB.\r\n%\r\n% I've written timing code many different times over the years,\r\n% using a variety of methods.  Along the way I've learned some\r\n% important tips and pitfalls from people like\r\n% <https:\/\/www.mathworks.com\/company\/aboutus\/founders\/clevemoler.html % Cleve Moler> and\r\n% <https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/loadAuthor.do?objectType=author&objectId=1095021 % Bill McKeeman>.  Late last year I was working on YABP (Yet\r\n% Another Benchmark Program), and I wondered if I could\r\n% encapsulate at least some of those tips into a timer utility\r\n% function, one that anyone could use without thinking too hard\r\n% or making too many choices.  The result was the function\r\n% |timeit.m|, which you can\r\n% <https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/loadFile.do?objectId=18798&objectType=FILE % find on the MATLAB Central File Exchange>.\r\n%\r\n% |timeit| takes one input argument and returns one output\r\n% argument:\r\n%\r\n%     s = timeit(f)\r\n%\r\n% |f| is a function handle, and |s| is the time, in seconds,\r\n% required to call that function handle.  |timeit| calls |f| with\r\n% no input arguments.\r\n% To use |timeit|\r\n% successfully, it's very helpful to know how to make\r\n% < % anonymous function handles>.  Here are a couple of examples:\r\n%\r\n% How much time does it take to compute |sum(A.' .* B, 1)|, where\r\n% |A| is 12000-by-400 and |B| is 400-by-12000?\r\n\r\nA = rand(12000, 400);\r\nB = rand(400, 12000);\r\nf = @() sum(A.' .* B, 1);\r\ntimeit(f)\r\n\r\n%%\r\n% How much time does it take to dilate the text.png image with\r\n% a 25-by-25 all-ones structuring element?\r\n\r\nbw = imread('text.png');\r\nse = strel(ones(25, 25));\r\ng = @() imdilate(bw, se);\r\ntimeit(g)\r\n\r\n%%\r\n% The function |timeit| is designed to handle three key timing\r\n% issues automatically:\r\n%\r\n% * \"Warming up\" the function to be timed, in order to eliminate\r\n% one-time effects related to file systems, memory systems,\r\n% M-file parsing and optimization, etc.\r\n% * Adjusting to the granularity of the timer functions |tic| and\r\n% |toc|.  |timeit| does this by calling |f| in an \"inner loop,\"\r\n% with the number of inner loop repetitions set so that each call\r\n% to |tic| \/ |toc| measures a time interval of at least about\r\n% 0.01 seconds.  The appropriate number of inner loop repetitions\r\n% is automatically determined.\r\n% * Running an \"outer loop\" that calls and times the inner loop\r\n% in a set of repeated trials.  The median time for the trials is\r\n% then used to compute the time estimate.\r\n%\r\n% Also, since the calls to |f| all occur within a function, the\r\n% large variability associated with timing code from the\r\n% command prompt is eliminated.\r\n%\r\n% The function is not perfect, and other people might make\r\n% different choices than I did.  For example, you could argue\r\n% that I should be using the minimum time instead of the median\r\n% time for the time trials.  I chose median because in my\r\n% experience I have found that it takes fewer repetitions to get\r\n% a repeatable result by using the median.  Also, |timeit| will\r\n% call |f| at least 13 times, which is possibly more than\r\n% necessary for some longer-running functions.\r\n%\r\n% If you are interested in issues and techniques related to\r\n% performance measurement, particularly in MATLAB, then I highly\r\n% recommend that you read\r\n% <https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/loadFile.do?objectId=18510&objectType=file % Bill's white paper on performance measurement>.\r\n%\r\n% If you have ideas about improving |timeit|, I'd be very\r\n% interested to hear them.\r\n##### SOURCE END ##### 934335e2751d48dab39f28daec0977a6\r\n-->","protected":false},"excerpt":{"rendered":"<p>\r\n\r\nWhen we work on improving the speed of image processing functions in MATLAB, we are naturally interested in getting timing\r\nmeasurements. And we know customers are often timing their own code,... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/steve\/2008\/02\/29\/timing-code-in-matlab\/\">read more >><\/a><\/p>","protected":false},"author":42,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[124,76,288,460,106,126,474],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts\/198"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/users\/42"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/comments?post=198"}],"version-history":[{"count":3,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts\/198\/revisions"}],"predecessor-version":[{"id":2776,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts\/198\/revisions\/2776"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/media?parent=198"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/categories?post=198"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/tags?post=198"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}