Comments on: The conv function and implementation tradeoffs https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/?s_tid=feedtopost Retired from MathWorks in 2024 after 30 years of service. Can now be found at MATLAB Central, https://hornjourney.com, and https://matrixvalues.com. MathWorks career included image processing, toolbox development, MATLAB development and design, development team management, MATLAB design standards, Steve on Image Processing blog (https://blogs.mathworks.com/steve). Co-author of Digital Image Processing Using MATLAB (https://www.imageprocessingplace.com/DIPUM-3E/dipum3e_main_page.htm). French horn enthusiast, member of Concord Orchestra and Melrose Symphony, member of the board of Cormont Music and the Kendall Betts Horn Camp. Tue, 29 Oct 2019 12:03:07 +0000 hourly 1 https://wordpress.org/?v=6.2.2 By: Steve https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22412 Fri, 04 Dec 2009 18:54:38 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22412 Brad—You and I are thinking about the conv function very differently. If I understand your comment, you are thinking of conv as an approximation to continuous-time convolution. I think of conv as implementing discrete-time convolution. It does this exactly; there are no approximations involved (other than the normal floating-point round-off issues). I’ve used discrete-time convolution about a gazillion times for signal and image processing in my work, and I don’t think I’ve ever used it to simulate or approximate continuous-time convolution. The FFT-based method here is mathematically equivalent to discrete-time convolution, so there are no approximation issues there.

]]>
By: Brad https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22395 Fri, 04 Dec 2009 00:28:41 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22395 one practical issue you might want to point users to is that when using conv if the data is not sampled at a high enough resolution then huge rounding errors can occur due to the numerical intergraton used in computing conv. I havent tested to see if the fft method does the same but it probably does. I usually interpolate the data by a factor of at least 4 before performing the conv function.

]]>
By: Steve https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22312 Tue, 10 Nov 2009 12:02:00 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22312 Francisco—Cache locality is always important, and changes in cache locality can certain change the location of the cross-over point in comparing the two algorithms. It won’t change the overall scaling, though.

Premature optimization is optimizating code before you know it is producing the correct answer, or before you know the code is actually a performance bottleneck.

]]>
By: Francisco https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22309 Tue, 10 Nov 2009 03:03:45 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22309 Does caching play a role in the speed of conv vs. fft_conv? It would seem that a clever order of multiplications would increase cache locality greatly in the conv function.

If not, do you even need to take cache locality into consideration for MATLAB, or is it just “premature optimization”?

]]>
By: Loren Shure https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22292 Thu, 05 Nov 2009 16:01:44 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22292 If you look towards the end of the fftfilt program, you will see that there’s a check to see if the inputs are real, an if so, to remove the imaginary part of the output, since it is only round-off. One of many things to do so users don’t get surprised.

–Loren

]]>
By: Steve https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22287 Tue, 03 Nov 2009 22:19:07 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22287 Mark—No problem about nextpow2. I’ve always thought that function name was confusing, and I usually look at the help for it whenever I use it.

About FFT complexity – yes, MATLAB uses O(L log L) algorithms for all sizes. Running time is fastest for powers of two (the constant of proportionality is smallest), but time scales with (L log L) for composite and even for prime L.

]]>
By: Mark Andrews https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22286 Tue, 03 Nov 2009 21:14:46 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22286 Well, it would help if I read the documentation, wouldn’t it? I thought nextpow2 returned the number 2^n, not the exponent n. My apologies.

]]>
By: Mark Andrews https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22285 Tue, 03 Nov 2009 21:11:44 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22285 Just some minor points, but I think

K = 2^nextpow2(L);

should be

K = nextpow2(L);

otherwise K is enormous :-)

Are you sure that the complexity of the FFT is O(L log L) regardless of L? I thought this was a lower bound for L a power of 2.

]]>
By: Steve https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22283 Tue, 03 Nov 2009 19:07:03 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22283 To all—After almost four years of writing this blog, I still have no ability to predict which topics will draw reader comment! ;-)

Peter—Most the comments people attempt to post here are desperate queries for help with homework or a graduate school research project (I don’t let those through), so yours is admirably on topic. ;-)

Your suggestion is a good one. We can look into making some changes so that imfilter does not need to pad. Just to set your expectations, though … the feature change deadline for R2010a was some time ago, and we’re already working on R2010b.

Jeremy and Chris—I tend to think of circular convolution and periodic boundary conditions as being basically the same issue, and it is not a reason to avoid using the DFT to compute convolution. Here’s the theory, for any reader who might be interested. (Note that I’m using mathematical notation here, not MATLAB syntax.)

If x[n] is nonzero only for 0 <= n <= P-1, and h[n] is nonzero only for 0 <= n <= Q-1, then the convolution y[n] = (x*h)[n] is nonzero only for 0 <= n <= L-1, where L = P + Q - 1. Further, y[n] for 0 <= n <= L-1 is exactly equal to the L-point circular convolution of x and h. So my conv_fft in this post produces exactly the same thing (except possibly for floating-point round-off effects) as conv. Overlap-and-add and similar methods were invented for a different reason. Consider the case where you want to filter a signal using FFT-based convolution, but you never have the entire signal at once. For example, you are building a system to process an input audio stream. Overlap-and-add allows you to perform FFT-based convolution on the input signal in chunks.

]]>
By: Cris Luengo https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22282 Tue, 03 Nov 2009 17:50:49 +0000 https://blogs.mathworks.com/steve/2009/11/03/the-conv-function-and-implementation-tradeoffs/#comment-22282 Also, going through the DFT you’re assuming a periodic boundary condition, whereas conv assumes zeros outside the boundaries. This can be significant, especially when using large convolution kernels.

I’m all for giving the user the option. It would be bad if you couldn’t control which method is being used.

]]>