Steve Eddins has developed MATLAB and image processing capabilities for MathWorks since 1993. He also coaches development teams on designing programming interfaces for engineers and scientists. Steve coauthored Digital Image Processing Using MATLAB.
When we work on improving the speed of image processing functions in MATLAB, we are naturally interested in getting timing
measurements. And we know customers are often timing their own code, typically by using the MATLAB functions tic and toc.
It turns out to be surprisingly difficult to get reliable, repeatable timing measurements on modern computers. This is true
for any program, not just MATLAB.
I've written timing code many different times over the years, using a variety of methods. Along the way I've learned some
important tips and pitfalls from people like Cleve Moler and Bill McKeeman. Late last year I was working on YABP (Yet Another Benchmark Program), and I wondered if I could encapsulate at least some
of those tips into a timer utility function, one that anyone could use without thinking too hard or making too many choices.
The result was the function timeit.m, which you can find on the MATLAB Central File Exchange.
timeit takes one input argument and returns one output argument:
s = timeit(f)
f is a function handle, and s is the time, in seconds, required to call that function handle. timeit calls f with no input arguments. To use timeit successfully, it's very helpful to know how to make anonymous function handles. Here are a couple of examples:
How much time does it take to compute sum(A.' .* B, 1), where A is 12000-by-400 and B is 400-by-12000?
A = rand(12000, 400);
B = rand(400, 12000);
f = @() sum(A.' .* B, 1);
How much time does it take to dilate the text.png image with a 25-by-25 all-ones structuring element?
bw = imread('text.png');
se = strel(ones(25, 25));
g = @() imdilate(bw, se);
The function timeit is designed to handle three key timing issues automatically:
"Warming up" the function to be timed, in order to eliminate one-time effects related to file systems, memory systems, M-file
parsing and optimization, etc.
Adjusting to the granularity of the timer functions tic and toc. timeit does this by calling f in an "inner loop," with the number of inner loop repetitions set so that each call to tic / toc measures a time interval of at least about 0.01 seconds. The appropriate number of inner loop repetitions is automatically
Running an "outer loop" that calls and times the inner loop in a set of repeated trials. The median time for the trials is
then used to compute the time estimate.
Also, since the calls to f all occur within a function, the large variability associated with timing code from the command prompt is eliminated.
The function is not perfect, and other people might make different choices than I did. For example, you could argue that
I should be using the minimum time instead of the median time for the time trials. I chose median because in my experience
I have found that it takes fewer repetitions to get a repeatable result by using the median. Also, timeit will call f at least 13 times, which is possibly more than necessary for some longer-running functions.
If you are interested in issues and techniques related to performance measurement, particularly in MATLAB, then I highly recommend
that you read Bill's white paper on performance measurement.
If you have ideas about improving timeit, I'd be very interested to hear them.