Steve on Image Processing

Contents

I want to thank Steve for letting me guest blog about some recent work we did to help our large data users work with their imagery in MATLAB using the Image Processing Toolbox.

Several releases ago, we had been hearing from customers that they had "really big" images that they couldn't work with because they were too large to load into memory. How big are these images? "VERY big", they assured us. They couldn't even LOOK at them, much less operate on them in MATLAB! They could do nothing! They were very frustrated. Well, over the never several releases we worked hard to provide better tools to help these customers out.

Over the next few weeks I'll discuss a series of (relatively) new features that have taken these users from hitting Out of Memory errors at every turn, to being able to visually explore their large imagery in a smooth, interactive tool, to being able to process their images incrementally from disc, all while working with arbitrarily large images of arbitrary file format! A much improved workflow!

For this example we'll be working with a (relatively) large TIFF file that ships with the Mapping Toolbox, boston.tif. This file is only about 37 MB in memory, so not problematic in itself, but the tools I discuss here will scale to any size image.

tiff_filename = 'boston.tif';
imfinfo(tiff_filename)

ans = 

                     Filename: 'C:\MATLABs\R2011a\toolbox\map\mapdemos\boston.tif'
                  FileModDate: '04-Jun-2007 17:12:10'
                     FileSize: 38729900
                       Format: 'tif'
                FormatVersion: []
                        Width: 4481
                       Height: 2881
                     BitDepth: 24
                    ColorType: 'truecolor'
              FormatSignature: [77 77 0 42]
                    ByteOrder: 'big-endian'
               NewSubFileType: 0
                BitsPerSample: [8 8 8]
                  Compression: 'Uncompressed'
    PhotometricInterpretation: 'RGB'
                 StripOffsets: [12x1 double]
              SamplesPerPixel: 3
                 RowsPerStrip: 256
              StripByteCounts: [12x1 double]
                  XResolution: 300
                  YResolution: 300
               ResolutionUnit: 'Inch'
                     Colormap: []
          PlanarConfiguration: 'Chunky'
                    TileWidth: []
                   TileLength: []
                  TileOffsets: []
               TileByteCounts: []
                  Orientation: 1
                    FillOrder: 1
             GrayResponseUnit: 0.0100
               MaxSampleValue: [255 255 255]
               MinSampleValue: [1 1 1]
                 Thresholding: 1
                       Offset: 38729291
             ImageDescription: '"GeoEye"'
                     DateTime: '2007:02:23 21:46:13'
                    Copyright: '"(c) GeoEye"'
           ModelPixelScaleTag: [3x1 double]
             ModelTiepointTag: [6x1 double]
           GeoKeyDirectoryTag: [24x1 double]
            GeoAsciiParamsTag: 'State Plane Zone 2001 NAD = 83|'

"I can't even LOOK AT my image!"

Our first foray into large data viewing was simply producing an "overview" style image. If your image is too large to load into memory then it is not possible to view it using imshow with any of the standard syntaxes. Before committing the time and resources necessary work with a large image (moving it, copying it, converting to a different format, etc) users just want to do a "sanity" check on the data.

To help them accomplish this task we provided (in release R2008a) a very simple option to the imshow function, the 'Reduce' parameter. When set to true, imshow will subsample a large TIFF file and display a reduced version of that image. The subsampling ratio is based on your reported screen resolution.

This parameter is only valid for TIFF images. We started with the TIFF format in the beginning because MATLAB has excellent support for incremental reading and writing of TIFF images using the Tiff class. We recognize that most users will not be working exclusively with TIFF imagery and will want to view files in other formats as well.

Now... I don't want to scoop my subsequent blog posts too much, but let's just say we hope that we have sufficiently "adapted" (<== hint) to these customers' needs.

Let's view a reduced version of our image.

imshow(tiff_filename,'Reduce',true,'InitialMagnification','fit');

You'll notice that if you zoom in using the figure zoom tool that you will not see higher resolution imagery. Here's a closer look at Fenway Park.

% Simulate interactive zoom
xlim([44 390])
ylim([2638 2858])

It's clear here that you are actually viewing a reduced-resolution version of the original image. This 'Reduce' parameter is intended for users who just want to get a "quick & dirty" overview of a large TIFF image, and is not appropriate for seeing details in large images.

"That's cute, but I need to see my ACTUAL image, not some decimated imposter!"

Next we wanted to provide some "real", industrial strength large image viewing tool. To provide the best interactive viewing experience possible, we must create a multiresolution version of large images so that we can pull in pixels from the appropriate zoom-level as you pan and zoom through the data set.

Most users will be familiar with this behavior; it's very similar to how online mapping tools work. As you navigate around online maps, you will see tiles of data at discrete zoom levels (resolutions) being assembled to make up the current map view.

To provide this feature we delivered a new function (in release R2009a), rsetwrite, which allows you to create a reduced resolution data set (R-Set) from a large TIFF or NITF file. Creating the R-Set can take some time depending on the size of the image, but it is a one-time, up front cost.

To create an R-Set, the syntax is very straightforward. Let's build an R-Set from our large boston.tif image.

rset_filename = 'boston.rset';
rsetwrite(tiff_filename,rset_filename);

Now that you have your R-Set file, you can simply view it using the Image Tool as you would any other supported image file:

   imtool(rset_filename)

Here's a screenshot:

Using the Image Tool's pan & zoom tools you can now zoom into Fenway Park and see that, using the R-Set, we are now displaying the full resolution image data.

Here's another screenshot:

With R-Sets, you can zoom and pan around images of ANY SIZE, and the Image Tool will pull in only the data that it needs. Navigation of large imagery has never looked so good!

"Ok yea that's not bad, but I need to do more than just view the data!"

There are still many issues that I have not covered here such as, "I need to do more than just view my images", and "I'm not using TIFF or NITF files, what now?". These are excellent questions that I'll discuss next time. Stay Tuned!

Get the MATLAB code

Published with MATLAB® 7.12