The mystery of the failing TIFF append loop
For today's post I'd like to introduce guest blogger Ashish Uthama, a MATLAB developer. He's got a story to share with you about a topic, TIFF import and export, that I know interests many blog readers.
I work in the Image and Scientific data team. Our team goal is to make it easy for you to work with various formats like TIFF, NetCDF and HDF5 to name a few. We recently came across an interesting issue with imwrite that Steve suggested would be of interest to readers of this blog. In fact, a comment on this blog was one of the data points in our investigation.
About three months ago, we started hearing about instances where the following piece of code would error out sporadically after completing a few loop iterations successfully:
imageData = ones([400 400 100]); tiffFile = 'file.tif'; for ind = 1:100 imwrite( imageData (:,:,ind), tiffFile,'WriteMode','append'); end
The error message was a basic file open error:
??? Error using ==> writetif at 100 Couldn't open 'file.tif' for writing.
We didn’t have much luck reproducing this issue initially. Sporadic failures are extremely frustrating for both users and developers. Imagine a complex program, processing hundred’s of images, erroring out sporadically with this message!
The first step was to analyze the failure itself. In each iteration, imwrite tries to open the file in read-write mode, appends an image to the TIF file and closes it. The error message indicates that at one of these iterations, imwrite was unable to open the file. This implies that either the permission changed or the file was locked otherwise for writing at that moment.
The second step was trying to figure out the cause for this lock. We were lucky with one of the first reports where the user associated the failure mode with an open Windows Explorer window pointing to the destination folder. His program did not error out when he closed the Windows Explorer.
We continue to investigate this behavior, but our working assumption now is that the OS locks the file while it gathers meta-data or creates a preview thumbnail when Windows Explorer is open. So far, the OS in the reported cases has been limited to Windows 7.
With the information we have as of this moment, we are not sure yet if a truly robust workaround exists other than closing the Windows Explorer window. A first defensive programming instinct might be to check for a lock before writing within imwrite. However, unless the check and the write operations are atomic, this approach would not be successful. The function (or MATLAB for that matter) would not be able to check efficiently if a file is genuinely unwritable or if it was locked momentarily for writing.
When we find an issue like this and have information which would help our users proceed with their work, our technical support team puts together a page with the details of the workaround. Here is the page our team created for this issue:
https://www.mathworks.com/support/solutions/en/data/1-DQPRHC/?solution=1-DQPRHC
You can also get to it from https://www.mathworks.com/support/ by searching for ‘imwrite loop’.
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.