Comments on: Automatic array growth gets a lot faster in R2011a https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/?s_tid=feedtopost Steve Eddins is an enthusiastic amateur French horn player. Also, he has developed image processing and MATLAB software for MathWorks since 1993. Steve coauthored the book <i>Digital Image Processing Using MATLAB</i>. Tue, 29 Oct 2019 20:36:41 +0000 hourly 1 https://wordpress.org/?v=6.2.2 By: João Henriques https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24221 Wed, 18 May 2011 19:16:05 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24221 This is great, there are always instances where the output size is unknown, for example the foobar function there could return a variable number of elements.

The trick I’ve used so far (and with great results, I might say) is to store results in a cell array and defer concatenation until after the loop.

y = cell(1, n);
for k = 1:n
    y{k} = foobar(k);
end
y = cat(1, y{:});

It still doesn’t help in the extreme case of a WHILE loop where you can’t guess the maximum number of iterations.

]]>
By: Steve https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24219 Wed, 18 May 2011 16:12:02 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24219 Oliver—I don’t know anything about that.

]]>
By: Oliver Woodford https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24218 Wed, 18 May 2011 15:56:17 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24218 Great! I once noticed that setting values in a sparse array to zero, one by one, in a loop is very slow. Has this been fixed? (I haven’t yet upgraded to test it myself).

]]>
By: Ben https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24217 Wed, 18 May 2011 06:22:31 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24217 It’s great to see this feature added, and long overdue. There are plenty of problems where it’s difficult or impossible to calculate the array size beforehand, especially combinatoric optimization problems such as tree searches. The solution there is of course to use size doubling but this new feature will save the hassle.

]]>
By: Steve https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24215 Tue, 17 May 2011 18:28:21 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24215 Tom—I agree. I have been annoyed that the C++ vector class handled this sort of thing better than previous versions of MATLAB.

]]>
By: tom https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24214 Tue, 17 May 2011 18:25:03 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24214 This is something that allocators in the C++ STL have done for approximately forever. I’m glad to see that MATLAB now has a reasonable allocation strategy.

]]>
By: Steve https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24213 Tue, 17 May 2011 12:49:54 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24213 Wes and Julian—I will try to post additional details later this week.

Gary—I completely disagree. When you don’t know the the size of the array at the beginning of the loop, it can be extremely cumbersome to implement automatic array growth yourself in order to make the implementation scalable. I know this because I’ve had to do it many times. Now I simply don’t have to worry about it because MATLAB is an even better array-oriented language than it already was.

Not preallocating was regarded as a poor coding practice because MATLAB didn’t handle it well. Now MATLAB handles it well.

Also, the feature works for array growth by repeated concatenation, something I didn’t mention in the post.

]]>
By: Gary https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24211 Tue, 17 May 2011 12:31:52 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24211 1000x faster is quite an achievement! Unfortunately, part of me can’t shake the feeling that it’s a lot of effort to expend to improve MATLAB support of what’s essentially poor coding practice.

]]>
By: Julian https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24207 Tue, 17 May 2011 10:08:48 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24207 Thanks Steve, interesting post.

One question always struck me is about pre-allocation of cell arrays. Mlint pointed out that my loop might be growing on each iteration, but when the variable in question is a cell array that holds heterogeneous data, I wondered how sizing it in advance really helped without knowing the size of the data to be put in the cells!

Julian.

]]>
By: Wes https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24206 Tue, 17 May 2011 02:58:52 +0000 https://blogs.mathworks.com/steve/2011/05/16/automatic-array-growth-gets-a-lot-faster-in-r2011a/#comment-24206 Hi Steve

Thanks for the post. I have a few questions:

– Does this feature work for cell arrays too?

– Is it tied to for loops, or can this behaviour be triggered anywhere an array is growing?

– Presumably this features works under the hood by growing the actual array in memory by more than 1 element at a time (such as doubling the allocated size each time it runs out of space). This means that upon finishing the loop, the memory allocated for the array will be usually be larger than necessary. Does this allocation automatically get ‘trimmed’ to the actual length once the loop finishes, or at any other time? Or do we potentially have the situation where an array has 2^25+1 elments but it keeps a 2^26 element allocation size?

]]>