Timing code in MATLAB

Posted by Steve Eddins, February 29, 2008

6 views (last 30 days) | 0 Likes | 8 comments

When we work on improving the speed of image processing functions in MATLAB, we are naturally interested in getting timing measurements. And we know customers are often timing their own code, typically by using the MATLAB functions tic and toc. It turns out to be surprisingly difficult to get reliable, repeatable timing measurements on modern computers. This is true for any program, not just MATLAB. I've written timing code many different times over the years, using a variety of methods. Along the way I've learned some important tips and pitfalls from people like Cleve Moler and Bill McKeeman. Late last year I was working on YABP (Yet Another Benchmark Program), and I wondered if I could encapsulate at least some of those tips into a timer utility function, one that anyone could use without thinking too hard or making too many choices. The result was the function timeit.m, which you can find on the MATLAB Central File Exchange. timeit takes one input argument and returns one output argument:

   s = timeit(f)

f is a function handle, and s is the time, in seconds, required to call that function handle. timeit calls f with no input arguments. To use timeit successfully, it's very helpful to know how to make anonymous function handles. Here are a couple of examples: How much time does it take to compute sum(A.' .* B, 1), where A is 12000-by-400 and B is 400-by-12000?

A = rand(12000, 400);
B = rand(400, 12000);
f = @() sum(A.' .* B, 1);
timeit(f)

ans =

    0.1125

How much time does it take to dilate the text.png image with a 25-by-25 all-ones structuring element?

bw = imread('text.png');
se = strel(ones(25, 25));
g = @() imdilate(bw, se);
timeit(g)

ans =

    0.0037

The function timeit is designed to handle three key timing issues automatically:

"Warming up" the function to be timed, in order to eliminate one-time effects related to file systems, memory systems, M-file parsing and optimization, etc.
Adjusting to the granularity of the timer functions tic and toc. timeit does this by calling f in an "inner loop," with the number of inner loop repetitions set so that each call to tic / toc measures a time interval of at least about 0.01 seconds. The appropriate number of inner loop repetitions is automatically determined.
Running an "outer loop" that calls and times the inner loop in a set of repeated trials. The median time for the trials is then used to compute the time estimate.

Also, since the calls to f all occur within a function, the large variability associated with timing code from the command prompt is eliminated. The function is not perfect, and other people might make different choices than I did. For example, you could argue that I should be using the minimum time instead of the median time for the time trials. I chose median because in my experience I have found that it takes fewer repetitions to get a repeatable result by using the median. Also, timeit will call f at least 13 times, which is possibly more than necessary for some longer-running functions. If you are interested in issues and techniques related to performance measurement, particularly in MATLAB, then I highly recommend that you read Bill's white paper on performance measurement. If you have ideas about improving timeit, I'd be very interested to hear them.

Published with MATLAB® 7.5