I'd like to welcome back guest blogger Ashish Uthama for today's post. Ashish, a developer on the Image Processing Toolbox team, posted here previously about the mystery of the failing TIFF append loop.

Contents

Brief introduction to the TIFF format

A TIFF file is, well, a Tagged Image File Format (TIFF). As the name implies, the file itself consists of a whole bunch of 'tags' and corresponding values. For example, all TIFF files are expected to have the ImageWidth and ImageLength tags, whose values are the width and length (height) of the image stored within.

A core feature of this format lies in how image data is organized in a physical file. TIFF allows for the image data to be stored in chunks as either 'Tiles' or 'Strips'. You could think of a strip as a tile whose width is the same as the image. The format stores offsets for each of these strips or tiles in an image header. This allows for efficient random access of any chunk in the image. The original TIFF format specification calls for use of 32 bit file offset values. I guess you can imagine where I am going with that bit of information. 32 bits implies a maximum value of 2^32 for any offset. Note that these are offsets from the very beginning of the file. In effect, the 32 bit requirement limits the size of the largest possible TIFF file to be under 4 gigabytes. This was fine for a long time (The earliest TIFF release was in 1986), but we now have more data producers hitting this limit.

Enter BigTIFF

The libtiff community, a loosely organized collection of volunteer software developers, maintains and develops the LibTIFF library. LibTIFF implements routines to manipulate TIFF files. Since version 4.0.0 of LibTIFF, they have included a variant of the TIFF format which uses 64 bits for the offsets instead of 32 bits. As you can imagine, this allows for really big TIFF files. This variant is appropriately called BigTIFF . Do note that this underlying change causes BigTIFF to an incompatible variant to the 32 bit offset TIFF format (now called the classic format). The first stable version of libtiff with BigTIFF support came out just in time for us to be able to qualify and test it to go out with MATLAB R2012b.

BigTIFF support in MATLAB

As I mentioned in the introduction, TIFF files have tags and values. This approach gives the format a lot of flexibility in the type of data it can hold, and yes, with great flexibility comes great complexity.

Reading BigTIFF file in MATLAB is no different from reading classic (32bit offset) version TIFF files. imread and all its TIFF related options work with BigTIFF. With BigTIFF files, the most relevant option is the 'PixelRegion' parameter. If a BigTIFF image file is too large to fit in system memory, you could use this parameter to load a smaller part of the image. imwrite can easily write out classic TIFF files. However, writing out BigTIFF files is a bit more involved. The key assumption with using imwrite is that the entire image data is available as a matrix in memory, something not necessarily true with BigTIFF. Tiff images can be complex and there are hundreds of possible combinations of the various tag values. In addition, the format enforces rules (see Tables 2, 3, 4 and 5) for legal combinations of tag values.

MATLAB provides a gateway to the LibTIFF library routines through the Tiff class. This interface supports over 60 tags, allowing MATLAB users to work with a wide range of TIFF files. Moreover, this interface also supports working with BigTIFF files. Here is a code snippet which uses the Tiff class to write a (albeit small) BigTIFF file:

imdata = imread('example.tif');
% 'w'  will create a classic TIFF file
% 'w8' will create a BigTIFF file
% This option is the only differentiator when writing to these two formats.
bt     = Tiff('myfile.tif','w8');

More information on the tags used below is here.

tags.ImageLength         = size(imdata,1);
tags.ImageWidth          = size(imdata,2);
tags.Photometric         = Tiff.Photometric.RGB;
tags.BitsPerSample       = 8;
tags.SamplesPerPixel     = size(imdata,3);
tags.TileWidth           = 128;
tags.TileLength          = 128;
tags.Compression         = Tiff.Compression.JPEG;
tags.PlanarConfiguration = Tiff.PlanarConfiguration.Chunky;
tags.Software            = 'MATLAB';

setTag(bt, tags);
write(bt,  imdata);
close(bt);

Using BigTIFF with BLOCKPROC (Image Processing Toolbox)

The ability to store and retrieve parts of the image as tiles makes TIFF images a perfect fit for the Image Processing Toolbox function blockproc. blockproc was designed to handle large data and also has a parallel mode to take advantage of the Parallel Computing Toolbox capability to run on large clusters. Read support for TIFF files is provided by imread under the hood, hence blockproc inherits the ability to read BigTIFF files out of the box. However, write support is limited to classic TIFF files. To enable blockproc to write out a BigTIFF file, we need a custom image adapter. Brendan blogged about this here sometime back. I put together a simple BigTIFF writer image adapter based on his blog post. It is also available on the File Exchange for download.

type bigTiffWriter.m

classdef bigTiffWriter < ImageAdapter
    %BIGTIFFWRITER - A basic image adapter to write Big TIFF files.
    %
    % A simple ImageAdapter class implementing a Tiff writer ImageAdapter
    % object for use with BLOCKPROC. 
    % - Tile dimensions must be multiples of 16.
    % - Only uint8, RGB input image data is supported.
    %
    % Based on "Working with Data in Unsupported Formats"
    % https://www.mathworks.com/help/toolbox/images/f7-12726.html#bse_q4y-1
    %
    % Example:
    %    %%
    %    % Set file names and obtain size information from input file.
    %    inFile        = 'example.tif';
    %    inFileInfo    = imfinfo(inFile);
    %    outFile       = 'out.tif';
    %    %%
    %    % Create an output TIFF file with tile size of 128x128 
    %    tileSize      = [128, 128]; % has to be a multiple of 16.
    %    outFileWriter = bigTiffWriter(outFile, inFileInfo(1).Height, inFileInfo(2).Width, tileSize(1), tileSize(2));
    %    %%
    %    % Now call blockproc to rearrange the color channels.
    %    blockproc(inFile, tileSize, @(b) flipdim(b.data,3), 'Destination', outFileWriter);
    %    outFileWriter.close();
    %    imshowpair(imread(inFile), imread(outFile),'montage');
    %
    % See also: blockproc, Tiff, Tiff/writeEncodedTile
    %
    
    %   Copyright 2013 The MathWorks, Inc.
    
    properties(GetAccess = public, SetAccess = private)        
        Filename;        
        TiffObject;
        TileLength;
        TileWidth;        
    end
    
    methods
        
        function obj = bigTiffWriter(fname, imageLength, imageWidth, tileLength, tileWidth)
            % Constructor
            
            validateattributes(fname,       {'char'},   {'row'});
            validateattributes(imageLength, {'numeric'},{'scalar'});
            validateattributes(imageWidth,  {'numeric'},{'scalar'});
            validateattributes(tileLength,  {'numeric'},{'scalar'});
            validateattributes(tileWidth,   {'numeric'},{'scalar'});
            
            if(mod(tileLength,16)~=0 || mod(tileWidth,16)~=0)
                error('bigTiffWriter:invalidTileSize',...
                    'Tile size must be a multiple of 16');
            end
            
            obj.Filename   = fname;
            obj.ImageSize  = [imageLength, imageWidth, 1];
            obj.TileLength = tileLength;
            obj.TileWidth  = tileWidth;
            
            % Create the Tiff object.
            obj.TiffObject = Tiff(obj.Filename, 'w8');
            
            % Setup the tiff file properties
            % See "Exporting Image Data and Metadata to TIFF files
            % https://www.mathworks.com/help/techdoc/import_export/f5-123068.html#br_c_iz-1
            %
            obj.TiffObject.setTag('ImageLength',   obj.ImageSize(1));
            obj.TiffObject.setTag('ImageWidth',    obj.ImageSize(2));
            obj.TiffObject.setTag('TileLength',    obj.TileLength);
            obj.TiffObject.setTag('TileWidth',     obj.TileWidth);
            obj.TiffObject.setTag('Photometric',   Tiff.Photometric.RGB);
            obj.TiffObject.setTag('BitsPerSample', 8);
            obj.TiffObject.setTag('SampleFormat',  Tiff.SampleFormat.UInt);
            obj.TiffObject.setTag('SamplesPerPixel', 3);
            obj.TiffObject.setTag('PlanarConfiguration', Tiff.PlanarConfiguration.Chunky);           
        end
        
        
        function [] = writeRegion(obj, region_start, region_data)
            % Write a block of data to the tiff file.
            
            % Map region_start to a tile number.
            tile_number = obj.TiffObject.computeTile(region_start);
            
            % If region_data is greater than tile size, this function
            % warns, else it will silently pad with 0s.
            obj.TiffObject.writeEncodedTile(tile_number, region_data);
            
        end
        
        
        function data = readRegion(~,~,~) %#ok<STOUT>
            % Not implemented.
            error('bigTiffWriter:noReadSupport',...
                'Read support is not implemented.');
        end
        
        
        function close(obj)
            % Close the tiff file
            obj.TiffObject.close();
        end
        
    end    
end

Let us run the included example:

Set file names and obtain size information from input file.

inFile        = 'example.tif';
inFileInfo    = imfinfo(inFile);
outFile       = 'out.tif';

Create an output TIFF file with tile size of 128x128

tileSize      = [128, 128]; % has to be a multiple of 16.
outFileWriter = bigTiffWriter(outFile, inFileInfo(1).Height, inFileInfo(2).Width, tileSize(1), tileSize(2));

Now call blockproc to rearrange the color channels.

blockproc(inFile, tileSize, @(b) flipdim(b.data,3), 'Destination', outFileWriter);
outFileWriter.close();
imshowpair(imread(inFile), imread(outFile),'montage');

This adapter can serve as a starting point to write more complicated writer adapters for your particular tag combination of the Tiff format. Do you work with large image data? Do you, or would you use BigTIFF for your work? Do let us know!

Get the MATLAB code

Published with MATLAB® R2013a

Published with MATLAB® R2013a