# Advanced: making a 2d or 3d histogram to visualize data density38

Posted by Doug Hull,

This short video makes a 2d histogram as an alternative to plotting data points and visually estimating where the most data is.

John A replied on : 2 of 38

Doug, I am a little confused…I assume x and y are just random coordinates…you then created xd, yd…what is this actually?? How did you do it? Then in linespace you use x,y again, but in interp you use xd,yd. I think I understand the concept, but I’m lost in the details. Can you post all the code so I can run what you did??

dhull replied on : 3 of 38

@John,

XD and YD are the random points. When used in the interp1 function, XD and YD are the raw data that is then interpolated to the nearest nicely spaced data points in XI.

Make sense?

Doug

John A replied on : 4 of 38

Doug, in the linspace functions then you should use xd,yd not x,y…was that just a typo?? If so all makes sense, thx

dhull replied on : 5 of 38

@John,

I can see where that would work too. For simplicity, I had hidden where the data came from. X and XD both had the same range, so it did not matter.

I had originally planned on showing where the data came, but in production the movie got too long, so it ended on the cutting room floor!

Good catch!

Doug

Joe Pohedra replied on : 6 of 38

Sorry, I’m lost. I could not follow what’s happening without the data set to reproduce the example. Fortunately for me, the hist3 function seems to do all the heavy lifting I need.

randx = randn(100000,1);
randy = randn(100000,1);
z = hist3([randx randy],[40 40]);
surf(z)

Douglas Neal replied on : 7 of 38

Doug,

This is a great code! I am also lost though. When I run the line

>> z = accumarray([xr yr], 1, [n n]);


I get the following error:

??? Error using ==> accumarray
Third input SZ must be a full row vector with one element for each column of SUBS.


Also, can you comment how this differs from hist3.m?

dhull replied on : 8 of 38

@Doug

Please post all the code so I can reproduce it by copying and pasting.

Doug

Douglas Neal replied on : 9 of 38

@dhull

Here is what I was doing to try and duplicate your code:


xd = randn(100000,1);
yd = randn(100000,1);

n=49;
xi = linspace(min(xd(:)),max(xd(:)),n);
yi = linspace(min(yd(:)),max(yd(:)),n);

xr = interp1(xi,1:numel(xi),xd,'nearest')';
yr = interp1(yi,1:numel(yi),yd,'nearest')';

z = accumarray([xr yr], 1, [n n]);

figure(2)
surf(z)



I copied everything save the declarations of xd, yd out of your example in the video. Thanks!

dhull replied on : 10 of 38

@Douglas,

z = accumarray([xr’ yr’], 1, [n n]);

You needed to transpose xr and yr so they are columns when you put them together. The way it was done made them a long row, not two columns. Works fine now!

Doug

Douglas Neal replied on : 11 of 38

@dhull

Thanks! Works great now — very nice code!

Doug Carter replied on : 12 of 38

Error in ==> testrun at 22
z = accumarray([xr’ yr’], 1,[n n]);

Is there anyway to get around this or what is the best way to figure out how to maximize the amount of data without excedding the memory limits?

Thanks,
Doug (Ohio State University)

dhull replied on : 13 of 38

@Doug,

Without more context, I can not really answer the question. Please contact http://www.mathworks.com/support with all the relevant files to reproduce, and we will take a look at it.

Thanks,
doug

Colin Norris replied on : 14 of 38

I’m trying to use this function (which I’ve modified below) but I’m getting an error

??? Error using ==> accumarray
First input SUBS must contain positive integer subscripts.


EngSpd & EngTorq are independant 24000 x 1 vectors
SpdBP is a 1×16 vector
TrqBP is a 1×21 vector

Any ideas on how to fix this issue?

Thanks
Colin


xd = EngSpd;
yd = EngTorq;

xi = SpdBP;
yi = TrqBP;

xr = interp1(xi,1:numel(xi),xd,'nearest')';
yr = interp1(yi,1:numel(yi),yd,'nearest')';

z = accumarray([xr' yr'], 1, [length(SpdBP) length(TrqBP)]);

figure(5)
surf(z)

dhull replied on : 15 of 38

@Colin,

I would put a break point on the line with accumarray. Once stopped in the debugger, figure out what the inputs to accumarray are. It looks like the first one is not what is expected.

Doug

Andrea replied on : 16 of 38

Thanks for the tutorial. Nice thing, I never came up against accumarray which seems to be very useful!

But actually there is a mistake when calculating the BIN-number: since interp1 assigns the nearest value in every dimension, all data points say in x-direction belonging to BIN 1 and half of them in BIN 2 will be assigned to (1,:). Correct should be interpolating to

xr = interp1(xi,0.5:numel(xi)-0.5,xd,'nearest')';
yr = interp1(yi,0.5:numel(yi)-0.5,yd,'nearest')';


which requires

z = accumarray([xr yr] + 0.5, 1, [n n]);


Best
Andrea

dhull replied on : 17 of 38

@Andrea,

Thank you for catching a subtlety there!

Doug

Thomas Smith replied on : 18 of 38

In the example code, the data from xd will be plotted on the Y axis, and yd on the X axis. To get a properly labeled graph, with the X data along the X axis, you actually need to say:

surf(xi, yi, Z');


Why is that? In the example, we have

Z = accumarray([xr yr], 1, [n n]);


For instance, if something falls in the 1st X-bin and the 5th Y-bin, it will get counted in Z(1,5). But, the MATLAB plotting commands like SURF and CONTOUR take their inputs in the opposite way: To have something show up at (1,5) in the graph, it should be in Z(5,1).

Hope this is useful to the next person who’s thinking sideways!
-Thomas

Thomas Smith replied on : 19 of 38

I made the changes suggested by Andrea and myself, and put them into a nice copy-pasteable (and editable!) file on GitHub:

https://gist.github.com/883933

Jbrand replied on : 20 of 38

I am trying to plot out a point spread function, there are positive and negative x and y values and I get this message

??? Error using ==> accumarray
First input SUBS must contain positive integer
subscripts.

any help?

dhull replied on : 21 of 38

@Jbrand,

You can not pass negative numbers to accumarray. Check your inputs.

Doug

Megan replied on : 22 of 38

Thanks for the video! I was able to use it effectively for 2 vectors (xd & yd).
I had some other questions about the extent to which I could use this code, or if there are other options available.
1. Is there an alternative to accumarray that will take negative inputs -or am I able to edit accumarray in a way that would allow it to take negative inputs?
2. Is there any way to have such a histogram plot for 3 vectors (xd, yd and zd)? What I would like to have is the x and y coordinates with z as the height and color coded by the frequency distribution. I am basically trying to visualize a special euclidean(2) space (since my z vector lies on polar coordinates). Is there any way to adapt your code to work for 3 vectors in this way?

I know these questions don’t directly apply to the working of the posted tutorial, but any suggestions would be very much appreciated!

Rod replied on : 23 of 38

I believe I have implemented the 3d analogy for use with a 3d color histogram. Correct me if this is wrong but it seems to be working. Hope this helps someone.

nbins = 16;
redmat = colorimage(:,:,1);
greenmat = colorimage(:,:,2);
bluemat = colorimage(:,:,3);
ri = linspace(0,255,nbins);
gi = linspace(0,255,nbins);
bi = linspace(0,255,nbins);
rtp = interp1(ri,1:numel(ri),double(redmat(:)),’nearest’);
gtp = interp1(gi,1:numel(gi),double(greenmat(:)),’nearest’);
btp = interp1(bi,1:numel(bi),double(bluemat(:)),’nearest’);
Z=accumarray([rtp,gtp,btp],1,[nbins,nbins,nbins]);

The only issue I have found is displaying this behemoth, since surf() only takes in mxn or mxnx3.

Catherine replied on : 24 of 38

Hi there, I would like to ask if you know how to label the peak values of the data points in your 3D plot. I can’t seem to find any commands in MATLAB that are able to perform that function. Thank you.

Doug replied on : 25 of 38

Check out the command >>annotation

Raj replied on : 26 of 38

Hi,

Thank you very much for this post. Its wonderful and exactly what I need to look some 3.5 million data points I have. So I have gone through each comments and details on this page and summarized the code as given below. But I am getting an error “Error using accumarray, First input SUBS must contain positive integer subscripts.” I know there are no negative values in xr and yr, and as suggested I am adding 0.5. So how can I fix it? Please help. Thanks in advance.

x = MAge000(:);
y = DSTS000(:);

% NaN values taken out by
x = x(isfinite(x(:, 1)), :);
y = y(isfinite(x(:, 1)), :);

% Number of bins
n = 49;
xi = linspace(min(x(:)), max(x(:)), n);
yi = linspace(min(y(:)), max(y(:)), n);

xr = interp1(xi, 0.5:numel(xi)-0.5, x, ‘nearest’);
yr = interp1(yi, 0.5:numel(yi)-0.5, y, ‘nearest’);

% [yr xr]: Y then X because the plotting commands take matrices that look like Z(y-coord, x-coord)
Z = accumarray([yr xr] + 0.5, 1, [n n]);

figure(1);
surf(xi, yi, Z);

Raj replied on : 27 of 38

Hi,
Never mind, I found my mistake, and fixed it. It should be

y = y(isfinite(y(:, 1)), :);

not

y = y(isfinite(x(:, 1)), :);

But I do have a question regarding the selection of n. Why n = 49 and not 50. May be its a stupid question but I dont know/understand. So if you can explain to me will be great.

Thanks again,

Raj

Doug replied on : 28 of 38

@Raj,

It is an arbitrary number.

Doug

Vipin Iyer replied on : 29 of 38

Hi Doug,
This might be a dumb question but before I spend the time to implement this, do you think this will work for an array having 2 million points?
Thanks
-Vipin

Doug replied on : 30 of 38

d = rand(1,2000000);
hist(d)

2 million is not a big deal to MATLAB.

Neda replied on : 31 of 38

Hi,
Thank you for your useful codes. My problem is that when I used the code there was an error:
??? Error using ==> interp1 at 125
X and Y must be of the same length.

when I run the codes line by line I noticed that the error happend because of this line in my program:

xr = interp1(xi,1,numel(xi),a,’nearest’)

I clicked on the error and it went through the function named interp1

Do you have any idea why this happened and what should I do?

I would be thankful if you could send me an email.
Regards

Doug replied on : 32 of 38

Make sure X and Y (The first two inputs) are vectors of the same length. See the doc for interp1.

Iain replied on : 33 of 38

@Thomas Smith, Andrea and Doug
I know this thread is a little old now, but i was wondering if you had any suggestions on keeping the heatmap colour consistent between plots. To elaborate, if I look at two different data sets using the code you outlined, will the colour be relative to the data set or an external control i.e. if i see yellow on one data set does this amount to the same yellow/density on another data-set or are they effectively arbitrary and cannot be compared? Thanks

Doug replied on : 34 of 38

You can look at clim. Just like you would hold ylim constant between to plots, same with clim (color limit)

Pinkee replied on : 35 of 38

Hi Doug,
Thanks for such great videos. They are truly of great help. I have a question that is, although not directly related to this video, is related to density functions. How would you plot the PDF of Z = sqrt(X.^2+Y.^2), if X and Y are uniformly distributed over (0,10). I tried using hist3, but I was not sure if that was the right way. I hope your advice will help me figure out. Thank you once again.

Doug replied on : 36 of 38

I am not a great stats guy, so not sure I can answer. Seems more stats than programming.

Doug

Claudia replied on : 37 of 38

Hi everyone,
I have found this code very useful while working with a 2D matrix.
Now, I wish I could reproduce the same results with a 3D matrix: I have a X,Y,Z matrix and I wish to have in a voxel/volume the accumulation of my events. I have tried few things with no success. Anyone can help me with this?
It does not really work If i use the accumarray by just adding one dimension!