Mike on MATLAB GraphicsGraphics & Data Visualization

Note

Mike on MATLAB Graphics has been retired and will not be updated.

This is machine translation

Translated by
Mouseover text to see original. Click the button below to return to the Original version of the page.

Memory Consumption8

Posted by Mike Garrity,

Today we're going to look at how MATLAB Graphics uses memory.

First we need a way to tell how much memory MATLAB is using. Because I'm using a Windows system, I can use the memory function. This isn't the most accurate way to measure memory consumption, but it is easy to use, and it is accurate enough when we're working with large arrays.

rng default
minit = memory;
mega = 1024^2;
sprintf('%g MB',minit.MemUsedMATLAB/mega)

ans =

1218.53 MB



Now we need a large array. I'm going to create an array with 150 million values, and I'll use the uint8 data type to keep things relatively compact. I'm using some fairly random looking data to make sure that we don't get confused by some optimizations which can kick in for certain types of simple curves.

npts = 150000000;
events = sort(randi(npts,[500 2]),2);
[~,ix] = sort(abs(events(:,2)-events(:,1)),'descend');
events = events(ix,:);
d = zeros([1 npts],'uint8');
for i=1:size(events,1)
d(events(i,1):events(i,2)) = randi(255,1);
end


We can see that this array consumes 150 million bytes, and MATLAB's memory grew by a little more than that.

whos d
marray = memory;
sprintf('%g MB',(marray.MemUsedMATLAB - minit.MemUsedMATLAB)/mega)

  Name      Size                       Bytes  Class    Attributes

d         1x150000000            150000000  uint8

ans =

152.117 MB



Now we'll create a plot from this array. First we'll create a figure and axes.

axes
drawnow
maxes = memory;
sprintf('%g MB',(maxes.MemUsedMATLAB - marray.MemUsedMATLAB)/mega)

ans =

26.4648 MB



That used a little more than 25 megabytes. That's because this is the first time we've used MATLAB Graphics in this session of MATLAB. Because of that, it had to load the graphics libraries, and get OpenGL started. Subsequent plots won't take this hit.

Now we'll call plot.

plot(d)
ylim([0 255])
drawnow


If we check on memory consumption, we'll see that MATLAB is now using another 20 megabytes or so.

mplot = memory;
sprintf('%g MB',(mplot.MemUsedMATLAB - maxes.MemUsedMATLAB)/mega)

ans =

23.4492 MB



What do you think will happen if we change one of the values in that array? Let's try it and find out.

d(1000) = 255 - d(1000);
mcow = memory;
sprintf('%g MB',(mcow.MemUsedMATLAB - mplot.MemUsedMATLAB)/mega)

ans =

143.109 MB



The amount of memory we're using went way up. That's because MATLAB arrays use something called "copy on write". When we first created the plot, the YData property of the line object was using exactly the same memory as the variable d. But when I modified d, the two arrays became "unshared". This means that d and the YData are each using 150 million bytes instead of sharing the same 150 million bytes.

But if we set the YData array to be equal to d ...

h = findobj(gca,'Type','line');
h.YData = d;
drawnow
mredraw = memory;
sprintf('%g MB',(mredraw.MemUsedMATLAB - mcow.MemUsedMATLAB)/mega)

ans =

-141.316 MB



... then we get that memory back.

In many cases, a better way to do this is to get rid of the variable in the workspace and modify the YData property of the line directly.

clear d
h.YData(1000) = 255 - h.YData(1000);
drawnow
mydata = memory;
sprintf('%g MB',(mydata.MemUsedMATLAB - mredraw.MemUsedMATLAB)/mega)

ans =

2.75781 MB



Keeping track of whether your arrays are sharing memory or not is often key to getting the best memory performance out of MATLAB graphics.

Here's another thing to look out for. In our example, we only supplied YData. In this case, the XData starts at 1 and increments by 1 for each data point. Because this is a very common case, the line object doesn't actually allocate any memory for the XData. But if we ask for the value of the XData property, it does allocate it. This means that something as simple as the following can be quite expensive.

disp(size(h.XData))
xdata = memory;
sprintf('%g MB',(xdata.MemUsedMATLAB - mydata.MemUsedMATLAB)/mega)

           1   150000000

ans =

1145.31 MB



What happened there is that it needed an array to pass to the size function, so it created one. And since the default datatype is double, it created a really big one! It's eight times the size of the array we originally passed in. That's pretty wasteful for an array which looks like this:

1 2 3 4 5 6 7 8 9 ...

So far we've been talking about the memory that is used by MATLAB's arrays. But there's actually a different type of memory that was getting used when we create the plot. MATLAB loads a copy of the geometry into the memory of the graphics card. It's important that the graphics card has a copy that is local, and is organized just right, so that it can redraw it as quickly as possible. A lot of the 20 megabytes that were used by the plot command were this type of memory.

But notice that this is quite a bit smaller than the original array. MATLAB Graphics generates a low-res version of the shape and sends that to the graphics card. But that low-res version has to contain enough data to capture all of those details in the shape. Getting this right is pretty tricky.

We call this step "level-of-detail" or "LOD". Level-of-detail algorithms are an important field of computer graphics. Unfortunately there isn't one simple algorithm which works for all types of geometry and in all types of situations. MATLAB Graphics currently has LOD algorithms for plot, line and image. We're working on improving those, and on adding more. But if you're working with one of the other types of graphics objects, then you won't be getting this LOD reduction, and MATLAB will be sending all of the geometry to the graphics card.

There actually is one other LOD algorithm in MATLAB. It's for the Patch object, and it is called reducepatch. At this point, we don't think that reducepatch is fast enough or reliable enough to build into the patch command so that it runs automatically. But you might want to use it when you are working with a really complex patch object.

If the data you're plotting is small, then we skip the LOD step and simply put all of the geometry into the memory of the graphics card.

Now lets save the plot to a FIG file.

fig = gcf;
fname = 'c:\temp\default.fig';
tic
savefig(fig,fname)
toc

Elapsed time is 102.143896 seconds.


That took more than a minute and a half, and if we look at the file, we'll see that it's more than 380 megabytes.

s = dir(fname);
sprintf('%g MB',s.bytes/mega)

ans =

386.894 MB



That's pretty big! There are a couple of reasons for this, but the biggest one is that it actually saved two copies. The reason for this is that in R2014b, the internals of MATLAB Graphics changed enormously. So if we want to create a FIG file which can be read by any version of MATLAB, we need to save a copy in the old format as well as a copy in the new format.

We can tell savefig not to do this by adding the compact flag.

If we do this, the save only takes about 1/2 as long.

tic
fname = 'c:\temp\compact.fig';
savefig(fig,fname,'compact')
toc

Elapsed time is 55.061655 seconds.


And we can see that the resulting FIG file is less than 1/2 as large.

s = dir(fname);
sprintf('%g MB',s.bytes/mega)

ans =

185.968 MB



That can be useful if you don't have any copies of MATLAB which are older than R2014b. The compact option is fine for saving and loading FIG files in newer versions, and it's much less expensive.

If you are using a version of MATLAB which is older than R2014b, you should know that there are a number of important differences in how MATLAB Graphics managed memory in older versions.

1. In older versions, MATLAB Graphics was loaded at MATLAB startup. This means that there isn't extra memory consumed on the first plot, but it means that MATLAB is using that memory even if you never do any graphics.
2. In older versions, MATLAB Graphics didn't support datatypes such as uint8 very well. As a result, it often copied the data into a double array, which consumed a LARGE amount of memory. This also meant that the "copy on write" memory sharing I described didn't occur.
3. In older versions, MATLAB Graphics was less aggressive about moving geometry into the memory of the graphics card. That was sometimes good in cases like this, where we're only drawing the data once. But it was a bottleneck in cases where we are repeatedly drawing the same data.
4. The compact option on savefig was introduced in R2014b because earlier versions only saved the one copy.

It's also important to know that the level-of-detail algorithms in MATLAB Graphics are continuing to evolve and get better. So the amount of memory consumed in that step should continue to get smaller in future releases.

I hope that this has given you some insight into how MATLAB Graphics uses memory. Hopefully it will give you some ideas about how to free up more memory for your computations.

Get the MATLAB code Published with MATLAB® R2015b

Eric replied on : 1 of 8
Hi Mike, nice article! Sometime would you be able to make some blog post(s) about the issues that customers are reporting about R2015b graphics performance? There are some good benchmark posts by folks such as Bruno, Rob and Matthew. It seems like part of the issue has to do with markers? I think if you made some posts explaining what's going on, and the potential timeline for improvements, it would make customers feel a lot better about the new graphics engine. Best, Eric
mgarrity replied on : 2 of 8
Yes, I'd love to, but there are an awful lot of different, unrelated cases being thrown around on that thread (which I actually haven't been following) and the apparently related thread on MATLAB Answers (which I have been following). I can't really answer all of them at the same time. I believe that a successful blog post about performance has to be pretty focused. They also take a lot longer to write than a lot of the other subjects I cover. That's because I need to collect a lot of data and check that it is really reliable. As for marker performance, that is an area that we're actively working on, but I really can't talk about what's coming in future releases. The current status is that as of R2014b, we have a single marker implementation which is used by all of the graphics objects. This helped the performance of objects like scatter which used to be second class citizens. But it hurt t he performance of the line object which used to have its own implementation. The old line marker implementation was very fast, but it was missing a lot of features which all of the other objects needed. As I said, we're working pretty hard at improving the performance of this implementation, and those improvements will be available in all of the graphics commands. One thing that you should be aware of with marker performance is that the '.' marker is special. This is the only marker type which OpenGL knows how to draw directly because it maps to GL_POINTS. All of the other markers require use to feed OpenGL a definition and then tell it to start drawing instances of that. This is where most of the performance issues are currently. So you might want to consider using the '.' marker when you're drawing a lot of them. The downside is that because '.' maps directly to an OpenGL primitive, it can be susceptible to problems in buggy OpenGL implemenations. For example, there's one popular card that draws them as little squares instead of little circles. We could work around that by making '.' work like 'o', but then we'd lose the performance advantages.
Eric replied on : 3 of 8
I understand Mike, thanks. A few thoughts/questions: 1) Is it possible to programmatically detect problematic drivers (for instance can you detect if a given feature is being emulated, like what Bruno ran into)? 2) Along with that, maybe there's some way to upload an 'HG2 graphics debugging toolkit' to the File Exchange, for people to run and provide the information to Support? Is there a way people could profile the memory transfers to/from the GPU, other CPU or GPU profiling tools that they could run, etc? 3) For the marker issue, would it be possible (or does HG2 already) use 'point sprites' for all marker types so they're all using point primitives? https://www.opengl.org/registry/specs/ARB/point_sprite.txt
mgarrity replied on : 4 of 8
> 1) Is it possible to programmatically detect problematic drivers (for instance can you detect if a given feature is > being emulated, like what Bruno ran into)? > The problem that I think we're hitting here is that the driver is claiming that it supports "feature X", when in reality it is emulating it in software. This case is pretty challenging. We've used blacklists for this sort of thing. They're basically lists of cards/drivers whose answers can't be trusted. > 2) Along with that, maybe there’s some way to upload an ‘HG2 graphics debugging toolkit’ to the File Exchange, > for people to run and provide the information to Support? Is there a way people could profile the memory transfers > to/from the GPU, other CPU or GPU profiling tools that they could run, etc? > We may do something like that. Our support team is currently trying out some tools like that, so getting repro steps for problematic cases to them can be really useful, both in terms of getting good answers, and in terms of helping them bash the tools. I'd really like to get make this stuff more accessible, but you currently need quite a bit of graphics knowledge to make sense of the results. > 3) For the marker issue, would it be possible (or does HG2 already) use ‘point sprites’ for all marker types so > they’re all using point primitives? https://www.opengl.org/registry/specs/ARB/point_sprite.txt > Yeah, the trick is coming up with an implementation that supports all of the features the various MATLAB users of markers need and also runs well on a wide variety of graphics cards. As you saw with Bruno's nVidia driver issue, every time we use a new feature like GL_ARB_point_sprite, we need to go through a pretty intensive qualification process to make sure that there isn't some graphics card that's going to make a dog's breakfast of it.
Eric replied on : 5 of 8
Yeah that all makes sense. The companies that make PC games go through very similar problems and process. Maybe everyone can get together and share blacklists :) But seriously, maybe some sort of user bench/troubleshooting tool could use timing information to detect emulated features? And even generate some plots, compare to stored reference images, flag detected issues and offer to send a report to TMW along with GPUinfo/driver/etc? That could let you build up a database of a lot of GPUs 'in the wild' for investigation/workaround.
mgarrity replied on : 6 of 8
That sounds like a really good idea! The SEGV reporting system that we've been using for a couple of releases now was our first step in that direction, and it has proven to be very valuable at finding patterns.
Dan replied on : 7 of 8
Mike, Thank you for the posts. I have two questions for you: 1. In the case of the memory consumption used when you query disp(size(h.XData)), is there a way to reclaim that memory? You're not creating a variable that you can clear. Will that memory usage remain until the figure is closed? Until Matlab is closed? 2. Is there a way to set savefig to always default to 'compact'? Thanks, Dan
mgarrity replied on : 8 of 8
You can set the XData to []. That will get rid of the memory. But it will remember that you've set that, so getting it won't recompute the value. To get all the way back to the default behavior, you also need to set XDataMode to auto. I don't know of a way to change the default on savefig. I'll ask around.

This site uses Akismet to reduce spam. Learn how your comment data is processed.