Comments on: The conv function and implementation tradeoffs http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/ Steve Eddins manages the Image & Geospatial development team at The MathWorks and coauthored Digital Image Processing Using MATLAB. He writes here about image processing concepts, algorithm implementations, and MATLAB. Fri, 10 Feb 2012 18:55:59 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Steve http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22412 Steve Fri, 04 Dec 2009 18:54:38 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22412 Brad—You and I are thinking about the conv function very differently. If I understand your comment, you are thinking of conv as an approximation to continuous-time convolution. I think of conv as implementing discrete-time convolution. It does this exactly; there are no approximations involved (other than the normal floating-point round-off issues). I've used discrete-time convolution about a gazillion times for signal and image processing in my work, and I don't think I've ever used it to simulate or approximate continuous-time convolution. The FFT-based method here is mathematically equivalent to discrete-time convolution, so there are no approximation issues there. Brad—You and I are thinking about the conv function very differently. If I understand your comment, you are thinking of conv as an approximation to continuous-time convolution. I think of conv as implementing discrete-time convolution. It does this exactly; there are no approximations involved (other than the normal floating-point round-off issues). I’ve used discrete-time convolution about a gazillion times for signal and image processing in my work, and I don’t think I’ve ever used it to simulate or approximate continuous-time convolution. The FFT-based method here is mathematically equivalent to discrete-time convolution, so there are no approximation issues there.

]]> By: Brad http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22395 Brad Fri, 04 Dec 2009 00:28:41 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22395 one practical issue you might want to point users to is that when using conv if the data is not sampled at a high enough resolution then huge rounding errors can occur due to the numerical intergraton used in computing conv. I havent tested to see if the fft method does the same but it probably does. I usually interpolate the data by a factor of at least 4 before performing the conv function. one practical issue you might want to point users to is that when using conv if the data is not sampled at a high enough resolution then huge rounding errors can occur due to the numerical intergraton used in computing conv. I havent tested to see if the fft method does the same but it probably does. I usually interpolate the data by a factor of at least 4 before performing the conv function.

]]> By: Steve http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22312 Steve Tue, 10 Nov 2009 12:02:00 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22312 Francisco—Cache locality is always important, and changes in cache locality can certain change the location of the cross-over point in comparing the two algorithms. It won't change the overall scaling, though. Premature optimization is optimizating code before you know it is producing the correct answer, or before you know the code is actually a performance bottleneck. Francisco—Cache locality is always important, and changes in cache locality can certain change the location of the cross-over point in comparing the two algorithms. It won’t change the overall scaling, though.

Premature optimization is optimizating code before you know it is producing the correct answer, or before you know the code is actually a performance bottleneck.

]]> By: Francisco http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22309 Francisco Tue, 10 Nov 2009 03:03:45 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22309 Does caching play a role in the speed of conv vs. fft_conv? It would seem that a clever order of multiplications would increase cache locality greatly in the conv function. If not, do you even need to take cache locality into consideration for MATLAB, or is it just "premature optimization"? Does caching play a role in the speed of conv vs. fft_conv? It would seem that a clever order of multiplications would increase cache locality greatly in the conv function.

If not, do you even need to take cache locality into consideration for MATLAB, or is it just “premature optimization”?

]]> By: Loren Shure http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22292 Loren Shure Thu, 05 Nov 2009 16:01:44 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22292 If you look towards the end of the fftfilt program, you will see that there's a check to see if the inputs are real, an if so, to remove the imaginary part of the output, since it is only round-off. One of many things to do so users don't get surprised. --Loren If you look towards the end of the fftfilt program, you will see that there’s a check to see if the inputs are real, an if so, to remove the imaginary part of the output, since it is only round-off. One of many things to do so users don’t get surprised.

–Loren

]]> By: Steve http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22287 Steve Tue, 03 Nov 2009 22:19:07 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22287 Mark—No problem about nextpow2. I've always thought that function name was confusing, and I usually look at the help for it whenever I use it. About FFT complexity - yes, MATLAB uses O(L log L) algorithms for all sizes. Running time is fastest for powers of two (the constant of proportionality is smallest), but time scales with (L log L) for composite and even for prime L. Mark—No problem about nextpow2. I’ve always thought that function name was confusing, and I usually look at the help for it whenever I use it.

About FFT complexity – yes, MATLAB uses O(L log L) algorithms for all sizes. Running time is fastest for powers of two (the constant of proportionality is smallest), but time scales with (L log L) for composite and even for prime L.

]]> By: Mark Andrews http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22286 Mark Andrews Tue, 03 Nov 2009 21:14:46 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22286 Well, it would help if I read the documentation, wouldn't it? I thought nextpow2 returned the number 2^n, not the exponent n. My apologies. Well, it would help if I read the documentation, wouldn’t it? I thought nextpow2 returned the number 2^n, not the exponent n. My apologies.

]]> By: Mark Andrews http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22285 Mark Andrews Tue, 03 Nov 2009 21:11:44 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22285 Just some minor points, but I think <pre> K = 2^nextpow2(L); </pre> should be <pre> K = nextpow2(L); </pre> otherwise K is enormous :-) Are you sure that the complexity of the FFT is O(L log L) regardless of L? I thought this was a lower bound for L a power of 2. Just some minor points, but I think

K = 2^nextpow2(L);

should be

K = nextpow2(L);

otherwise K is enormous :-)

Are you sure that the complexity of the FFT is O(L log L) regardless of L? I thought this was a lower bound for L a power of 2.

]]> By: Steve http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22283 Steve Tue, 03 Nov 2009 19:07:03 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22283 To all—After almost four years of writing this blog, I still have no ability to predict which topics will draw reader comment! ;-) Peter—Most the comments people attempt to post here are desperate queries for help with homework or a graduate school research project (I don't let those through), so yours is admirably on topic. ;-) Your suggestion is a good one. We can look into making some changes so that imfilter does not need to pad. Just to set your expectations, though ... the feature change deadline for R2010a was some time ago, and we're already working on R2010b. Jeremy and Chris—I tend to think of circular convolution and periodic boundary conditions as being basically the same issue, and it is not a reason to avoid using the DFT to compute convolution. Here's the theory, for any reader who might be interested. (Note that I'm using mathematical notation here, not MATLAB syntax.) If x[n] is nonzero only for 0 <= n <= P-1, and h[n] is nonzero only for 0 <= n <= Q-1, then the convolution y[n] = (x*h)[n] is nonzero only for 0 <= n <= L-1, where L = P + Q - 1. Further, y[n] for 0 <= n <= L-1 is exactly equal to the L-point circular convolution of x and h. So my conv_fft in this post produces exactly the same thing (except possibly for floating-point round-off effects) as conv. Overlap-and-add and similar methods were invented for a different reason. Consider the case where you want to filter a signal using FFT-based convolution, but you never have the entire signal at once. For example, you are building a system to process an input audio stream. Overlap-and-add allows you to perform FFT-based convolution on the input signal in chunks. To all—After almost four years of writing this blog, I still have no ability to predict which topics will draw reader comment! ;-)

Peter—Most the comments people attempt to post here are desperate queries for help with homework or a graduate school research project (I don’t let those through), so yours is admirably on topic. ;-)

Your suggestion is a good one. We can look into making some changes so that imfilter does not need to pad. Just to set your expectations, though … the feature change deadline for R2010a was some time ago, and we’re already working on R2010b.

Jeremy and Chris—I tend to think of circular convolution and periodic boundary conditions as being basically the same issue, and it is not a reason to avoid using the DFT to compute convolution. Here’s the theory, for any reader who might be interested. (Note that I’m using mathematical notation here, not MATLAB syntax.)

If x[n] is nonzero only for 0 < = n <= P-1, and h[n] is nonzero only for 0 <= n <= Q-1, then the convolution y[n] = (x*h)[n] is nonzero only for 0 <= n <= L-1, where L = P + Q - 1. Further, y[n] for 0 <= n <= L-1 is exactly equal to the L-point circular convolution of x and h. So my conv_fft in this post produces exactly the same thing (except possibly for floating-point round-off effects) as conv.

Overlap-and-add and similar methods were invented for a different reason. Consider the case where you want to filter a signal using FFT-based convolution, but you never have the entire signal at once. For example, you are building a system to process an input audio stream. Overlap-and-add allows you to perform FFT-based convolution on the input signal in chunks.

]]> By: Cris Luengo http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22282 Cris Luengo Tue, 03 Nov 2009 17:50:49 +0000 http://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22282 Also, going through the DFT you're assuming a periodic boundary condition, whereas conv assumes zeros outside the boundaries. This can be significant, especially when using large convolution kernels. I'm all for giving the user the option. It would be bad if you couldn't control which method is being used. Also, going through the DFT you’re assuming a periodic boundary condition, whereas conv assumes zeros outside the boundaries. This can be significant, especially when using large convolution kernels.

I’m all for giving the user the option. It would be bad if you couldn’t control which method is being used.

]]>