I’ve been trying hard to find a way for vectorizing relational matrix manipulation in Matlab.
Maybe there is an even simpler way that I am not aware of, but your #4 works.
I used it with
indices = sub2ind(size(A),rnotA+1,cnotA+1);

Note +1. I basically needed to manipulate the i+1,j+1 neighbors of elements satisfying a condition without relying on a for loop.

Thanks again!

-Nickitas

Logical indexing is nice, but not always practical. We produce hundreds of GB of data every day a subsequently load it into MATLAB. It’s possible to construct a short array of numerical indices, which contains the subset of the data I may be interested in but keeping a logical index around is not really feasible, either in MATLAB or on disk.

Further, to the comparison of logical vs. numerical indexing (on a smaller footprint example)

x = rand(2 * 1e9, 1); %16GB of data

tic; l = x > 0.5; toc
Elapsed time is 4.771542 seconds.

tic; i = find(l); toc
Elapsed time is 29.321260 seconds.

>> tic; y=x(l); toc
Elapsed time is 49.782177 seconds.

>> tic; z=x(i); toc
Elapsed time is 51.138457 seconds.

As you can see, there is very little in it for a large set, since most of the computation is likely to be memory bound. I have to say that I’m disappointed with the implementation of the numeric subscript. It is not surprising that a logical index must grow the resulting array, since it doesn’t know in advance how many elements may be in the result. However, to see reallocations on my system monitors during the final subscript call seems unnecessary, since I know *exactly* how many elements there’ll be. Memory reallocation would actually be the reason why, for large datasets, I’d expect the numerical index to outperform the logical index.

On a similar note, here are two features I’d like to see in MATLAB:
1) Don’t construct the logical array if you don’t need to: For example,

i = find(logical expression);

or even

b = logical expression;
i = find(b);
% b no longer used after here

There seems no reason why you have to do construct the temporary logical array first and then follow the computation by another pass over the logical array. This is slow and wastes an unnecessary amount of memory for a temporary.

2) Again, suppose I have a large dataset (200+ GB in size). I want to perform a computation on a subset of it (say 50GB). I have to make a copy to use the data in the computation, which is a problem, as I’m already riding hard against the amount of available memory. If I have a list of the indices already, I’d like to use an “indirect” array – an object that holds both indices and the original data and allows me to subscript it as a regular array. To preempt objections: a) Yes, I could write a class to do it but that’s not very fast. b) The object could be read only, so that you wouldn’t have to worry about multiple numerical indices with the same value.

Many thanks,

Tom

Here’s the documentation for end:

http://www.mathworks.com/access/helpdesk/help/techdoc/ref/end.html

The capability has been allowed, I believe, since its introduction as an index (and overloadable with the class system).

–Loren

Thanks very much for this solution…

B = A(setdiff(1:end, R), setdiff(1:end, C));

Steve - very neat,
Loren - I had no idea I could use “end” in a function call !

…where’s that documented ?

What I find interesting is how common a problem it is for (not necessarily beginner) users to run into the issue mentioned in Method #1. Often users are using subscripting and don’t realize they need to convert to linear indices to solve a specific problem.

I find myself having to repeatedly explain the difference and am glad you have addressed the topic here.

Nice reply to Tony’s challenge. You can use end’s in the expressions of your first solutions as well:

  A = reshape(1:49, 7, 7);
  R = 1:2:7; % Rows to remove
  C = 2:2:7;
  B = A(setxor(R, 1:end), setxor(C, 1:end));
  % or
  B = A(setdiff(1:end, R), setdiff(1:end, C));

–Loren

With regard to removing rows R and columns C from a matrix, you can use:

A = reshape(1:49, 7, 7);
B = A; % for comparison
R = 1:2:7; % Rows to remove
C = 2:2:7;
B(R, :) = [];
B(:, C) = [];

or:

A = reshape(1:49, 7, 7);
R = 1:2:7; % Rows to remove
C = 2:2:7;
B = A(setxor(R, 1:7), setxor(C, 1:7));
% or
B = A(setdiff(1:7, R), setdiff(1:7, C));

As for the arithmetic/logical combination, that will work … as long as all your elements of A are finite.

A = 1:1000;
B = A.*(A > (17^2)/2);
all(B == A | B == 0)

A = 1:1000;
A([5 982]) = NaN;
A([7 562]) = Inf;
A([324 870]) = -Inf;
B = A.*(A > (17^2)/2);
all(B == A | B == 0)

Note that not only are elements 5 and 982 of the second B NaN (as you would expect, since any arithmetic operation on a NaN results in a NaN result) but so are elements 324 and 870. This is because 0*Inf and 0*(-Inf) also both return NaN.

When I’m dealing with very large data sets, this is one of the most frustrating slow points, since it can exceed the time requirements of many of the calculations.
Dan

I often rely on the rather naughty mix of arithmetic and logical operations:

A .* (A > (17^2)/2)

…because it doesn’t require an assignment statement and can be used inline as part of a longer expression.

-Tony