In the past, when you opened a file in a MATLAB program, you had to keep track of it so you could close it under all possible conditions, i.e., if reading or processing failed, worked partially, or worked perfectly. You could achieve this using the catch block after a try. But you still might have to know before closing the file if everything you wanted to do was complete. Code could easily get awkward and buggy if there were lots of code paths to watch out for. With R2008a, you can use the onCleanup class to help you out more cleanly.
Contents
Example Code - Open File
Here's the start of a file I've written that opens up an image file for reading, using fopen.
type openImageFile0
function openImageFile0
% openImageFile Open an image file.
fd = fopen('pout.tif');
If I run it, you can see that I never close the file.
openImageFile0()
fids = fopen('all')
filename = fopen(fids(1));
filename(end-30:end)fids =
3
ans =
toolbox\images\imdemos\pout.tif
Tidy up now.
fclose(fids(1));
Example Code - Manage Open, Close, and Errors Using try/catch
Now let's look at another version of the code that will close the file it opens before returning.
type openImageFile1
function openImageFile1
% openImageFile1 Open/process file while ensuring file is
% closed on finish.
try
fd = fopen('pout.tif');
if fid == -1
error('LS:NoImageFile','no such file')
end
doMoreHere(fd);
catch Ex
disp('Can not read file')
end
if fid ~= -1
fclose(fid);
end
You can see that, even with try/catch, since I always want to close the image file, I have some extra logic at the end of the file. And I have to be careful to know whether the error occurs because the file was never opened, or if instead the error occurs in later processing.
Example Code - Use onCleanup to Ensure File Closes
Here's a file roughly equivalent to the first one, but it ensures that the image file is closed when the function is finished running.
type openImageFile2
function openImageFile2
% openImageFile2 Open an image file for further processing.
% Note: this doesn't do what you want, since the file will
% be closed after completing and the file identifier won't
% be active for use upon completion.
fd = fopen('pout.tif');
C = onCleanup(@()fclose(fd));
To see that this works we first check that no files are open, run the function, and make sure that no files are open afterwards.
fb = fopen('all'); openImageFile2() fp = fopen('all'); noFilesOpen = isequal(fp,fb,[])
noFilesOpen =
1
Example Code - Use onCleanup and Process After Opening the File
Let's see the next version of the file.
type openImageFile3
function fd = openImageFile3
% openImageFile3 Open an image file for further processing.
% Note: this doesn't do what you want, since the file will
% be closed after completing and the file identifier won't
% be active for use upon completion.
fd = fopen('pout.tif');
C = onCleanup(@()fclose(fd));
This function returns the file identifier, but as we'll see, the file is closed once the function returns. This means that any processing/reading we were planning to do with the file after calling the open function will not work (since the file will no longer be open).
fb = fp;
fid = openImageFile3()
fp = fopen('all');
noFilesOpen = isequal(fp,fb,[])fid =
3
noFilesOpen =
1
Even though fid is returned from the function call, the file is no longer open. This is because the onCleanup object C is cleared from existence once openImageFile3 completes execution.
Example Code - Use onCleanup and Process and Close File on Completion
Here's the last version of the file for this blog. As long as you call it with 2 outputs, you are good to go.
type openImageFile4
function [fd, C] = openImageFile4
% openImageFile4 Open an image file for further processing.
% Note: this does what you want, provided you call the
% function with both output arguments. When you are done
% cleaned up and the onCleanup object will trigger the file
% closing.
% Don't even open file if function isn't called with 2 outputs.
if nargout < 2
error('LS:MustHave2Outputs','must call with 2 outputs')
end
fd = fopen('pout.tif');
C = onCleanup(@()fclose(fd));
This version returns the file identifier and the onCleanup object. Once you have processed your file, simply clear the onCleanup object and the file will close.
To see how this might work, we call it from our function that does the processing.
type run4
function y = run4
% run4 processes an image file.
[fid,C] = openImageFile4();
blocksize = 100;
y = [];
while something
dat = fread(fid,count);
y = doSomething(y,dat);
something = update(something);
end
You can see that run4 calls openImageFile4, and then does the required processing. Because C will vanish once run4 completes, the file will be closed.
Other Uses?
The onCleanup object allows you to more cleanly manage resources such as closing files, closing windows (e.g., last week someone mentioned managing instances of waitbar), managing switching directories. What uses do you foresee for the onCleanup object in your work? Let me know here.
Get
the MATLAB code
Published with MATLAB® 7.6
Maybe this will grow on me, but I must admit, it seems somehow sloppy to set things up so that a job like closing the file is called implicitly by having an object disappear because the function whose scope it belongs to ends. Also, as I’m looking back through this blog, how does this help us deal with the possibilities of errors cropping up? Suppose that either 1. pout.tif doesn’t exist, or 2. you run into a problem in trying to use it. Does the onCleanup somehow save us from throwing an exception in your last example?
Thanks,
Dan
I’ve got several comments on this topic.
First, I can see a general benefit to this type of function since it protects from a general type of error. Although, part of me is wondering why the fundamental design of the MatLab language doesn’t provide adequate protection. Maybe Dan’s comment is related to this issue.
Does this command require the “in-line” generation of the function handle? Or can you generate the function handle on a previous line. If so, does this eliminate the empty function argument? Again if so, can you modify your openImageFile2 code to illustrate the separate line function handle?
Finally, since function handles are a central issue in this column, you can be sure to generate criticism from me regarding MatLab’s documentation on function handles. The FRP for such a complex topic needs dozens of examples, not just two. Further, the FRP ignores the topic of an argument list for a non-anonymous function. Finally, why do the smart folks at The MathWorks think it is useful that MatLab, “. . . provides a means of calling a function indirectly.” It seems that sound programming technique would be to avoid anything that is indirect.
Dan-
In the last example, there is not enough error checking yet so an exception can still be thrown (I was trying to illustrate something else though, about triggering the close of the file).
–Loren
Oliver-
You can generate the function handle however you want. It simply must be one that requires no input arguments. The reason a regular function handle isn’t mentioned on the FRP is because of requiring that all input arguments are dealt with to the function being called.
–Loren
It seems strange that (a task like) closing a file that is no longer used needs to be handled explicitly by the programmer at all. In Matlab programs, when data is no longer needed (i.e. when a variable holding a matrix goes out of scope), Matlab reclaims memory automatically. I think this would lead most new Matlab users to assume that the same mechanism would be used for other resources, such as file handles. I don’t see why Matlab couldn’t simply implement this behaviour: when a file handle goes out of scope, the file should automatically be closed.
I realise that changing the behaviour of fopen would break existing programs—not a reasonable solution. I would advocate deprecating fopen and replacing it with a function that returns an object that, when it goes out of scope, automatically closes the file.
The new fopen would require the returned handle to be assigned to a variable (in the way that, I suspect, onCleanup requires one to assign its return value to a variable) so that Matlab can detect when it goes out of scope.
While onCleanup offers a mechanism for programmers to specify their own cleanup behaviour, I think it is unnecessary. If Matlab could automatically reclaim resources, such as by closing files as described above—as it already does with most other built-in data types—cleanup is just object destruction, which is also provided by Matlab.
Interesting thoughts, C Rose. Unfortunately, we have too many people with too much code written to change the way fopen works now. But your point is well taken — about automatic cleanup when we can. I still think onCleanup is useful for times when you want figure windows to close, for example. There’s no guaranteed way to know the user is done with them, unless the application itself knows.
–Loren
Interesting idea, but I have to agree with the sentiments above that this isn’t the cleanset of solutions. The auto-cleaning file handle would be a great addition, though.
Anyways, when I read the post I thought about my mex files that I quite frequently use and which frequently remain in memory untill the next ‘clear all’, consuming large ammounts of memory. Is there a way to use this onCleanup trick to unload a mex file?
–DA
Daniel-
The unloading of mex would have to me from something other than the mex-file itself, I think. If it was called from a MATLAB M-file, you could do something like:
function fred C = oncleanup(@()clear('callMex') callMexWhen the call to fred finishes, the mex-file callMex will be cleared.
–Loren
As I read the Matlab documentation the implementation of onCleanup is fatally flawed and I don’t think anybody should use it unless some things can be clarified.
According to
help for onCleanup
‘the lifecycle of an object ends *sometime* between the time its variable is no longer used inside the function and the end of the function’
The implementation of onCleanup seems to rely on a subtly different interpretation of object lifecyle.
‘the lifecycle of an object ends at the end of the function within which it’s last reference is lost’
This means the following example
function y = run4
% run4 processes an image file.
[fid,C] = openImageFile4();
blocksize = 100;
y = [];
while something
…dat = fread(fid,count);
…y = doSomething(y,dat);
…something = update(something);
end
is broken.
According to Matlab spec C could be deleted any time after it appears in the line
[fid,C] = openImageFile4();
thus deleting the file handle whilst it was being used. It looks like a classic race condition problem.
To be safe you could rewrite the code
function y = run4
% run4 processes an image file.
[fid,C] = openImageFile4();
blocksize = 100;
y = [];
while something
…dat = fread(fid,count);
…y = doSomething(y,dat);
…something = update(something);
end
C;
But somehow I see the Matlab optimiser at some time in the future stripping out such dead code and rendering the trick useless anyway.
Stylistically it is also not obvious to a reader of the code what C is being used for and it’s inclusion in the code could easily be deleted by an eager maintainer unless the ode is accompanied by copious documentation.
There are however neater ways to solve the same problem in other languages.
The Python way:
http://docs.python.org/whatsnew/pep-343.html
with open(’/etc/passwd’, ‘r’) as f:
….for line in f:
……..print line
lexical scoping is used to clarify the required lifetime of the file handle f. As well if any exception is thrown from within the block the file is also closed.
The Ruby way:
File.open ‘/etc/passwd’ do |f|
…f.each_line do |l|
……puts l
…end
end
Very similar to the python way and again there is no requirement by the user to provide any cleanup or close functionality. In case of normal block termination or in the event of an error the open function will *always* close the file handle when the block terminates.
The onCleanup functionality could also be dealt with by adding an ensure keyword to the Matlab language as has Ruby and Python.
function do_open(file, flags, action)
… fid = fopen(file, flags)
… try
…….action(fid)
….ensure
…….fclose(fid)
….end
end
Notice there is no catch above. We don’t want to catch an error we just want to *ensure* that we cleanup if there is an error or not. Any error is propagated to code further upstream to handle. The ensure can be substituted with a catch and rethrow but is a bit uglier as you need to write code to handle closing the file if and if not an error occurs.
Using do_open in Matlab would be like
do_open(’/etc/passwd’, ‘r’, @body)
function body(fid)
… read(fid, 10)
… function_with_error()
end
As is obvious the function definition is kind of superfluous and would be nicer with some kind of multi line anonymous function
do_open(’/etc/passwd’, ‘r’ ) @(fid)
… read(fid, 10)
… function_with_error()
end
Brad Phelan
Loren, you say that you can’t change the way fopen works because that would break too much existing code. MATLAB has changed a lot over the years, breaking large chunks of my code on occasion. It’s very difficult trying to keep code compatible with older versions of MATLAB. For example, LOGIC arrays no longer being numeric was a *huge* change. Compared to that, changing the file pointers from a scalar integer value to an object seems rather trivial. Most code would still work the way it does now, especially if you can test for validity of the file pointer by examining its the numeric value (i.e. f>0).
Here’s one more vote for the self-closing file handle!
Daniel:
You can unload your MEX-files with ‘clear functions’.
Brad,
onCleanup objects obey the same rules as MATLAB objects do, since onCleanup is implemented as a MATLAB object. It actually has a rather trivial implementation, which you can see by looking at onCleanup.m.
I think you stated it very well when you said, “The lifecycle of an object ends at the end of the function within which its last reference is lost.” This is mostly true for MATLAB objects, including onCleanup objects. Objects may be deleted sooner explicitly with the delete or clear functions, or implicitly by reusing the variable containing the last reference to the object.
For typical uses, onCleanup objects will not be returned as output arguments from the functions which create them. Also, most uses will not involve explicit or implicit early destruction of onCleanup objects. Thus, the lifecycle of the onCleanup object will be exactly the scope of the function which contains it.
This is what Loren’s run4 would look like if the creation of the onCleanup object were not encapsulated in a separate function:
function y = run4 % run4 processes an image file. fid = fopen('pout.tif'); C = onCleanup(@()fclose(fid)); blocksize = 100; y = []; while something dat = fread(fid,count); y = doSomething(y,dat); something = update(something); end end-=>J
Hi Jason,
From the Matlab Documentation
http://tinyurl.com/2drurc
(Object Lifecycle) Inside a Function.
The lifecycle of an object referenced by a local variable or
input argument is defined to exist from the time the
variable is assigned until the time it is reassigned,
cleared, or no longer referenced within that function or any
handle array. If the variable is not cleared or reassigned,
the lifecycle of an object ends sometime between the time
its variable is no longer used inside the function and the
end of the function.
Within the time between a variable’s
last use and the end of function execution, the MATLAB
runtime destroys the object to which the variable refers
The key point is the last sentence. The Matlab runtime is allowed to delete an object before the end of the function. That is the object may be deleted before the function has finished executing. If this occured ( and it probably doesn’t at the moment) then the oncleanup implementation will fail.
Either the documentation is wrong or the implementation of onCleanup makes an assumption which may not be future proof. Personally I think the documentation makes sense and it gives Matlab future flexibility for gc optimization.
Regards
Brad Phelan
Brad,
It is true that a garbage collection algorithm could delete an unused variable once the last use has passed. There isn’t currently such an algorithm in MATLAB, but if you want to protect against a future implementation of one, you may explicitly delete or clear the onCleanup object, which will define a use of the variable.
This happens to make for rather readable code, actually:
function y = run4 % run4 processes an image file. fid = fopen(’pout.tif’); C = onCleanup(@()fclose(fid)); blocksize = 100; y = []; while something dat = fread(fid,count); y = doSomething(y,dat); something = update(something); end C.delete; end-=>J
The issue is not whether it works or not or that it can be made to work through a series of hacks. The problem is that the documentation is inconsistent. Either the documentation on object lifecyle should be changed to insist that objects last until the end of a function block or the implementation or documentation of onCleanup needs fixing to be consistent. Sorry to be picky but this is a feature promoted in a high profile blog and should be guaranteed to work in all future Matlab releases.
I notice that Octave ( the open source Matlab clone ) has a feature called unwind_protect.
http://www.cheric.org/education/eduaids/octave/octman/octave_41.html
unwind_protect
…while true
…end
unwind_protect_cleanup
…fprintf (’caught interrupt\n’);
end
which behaves the same as ensure in python and ruby. The code inside unwind_protect_cleanup will run regardless of event of an exception in the unwind_protect block. Even a ^C keypress will force the cleanup code to run.
It is also a clear and unambiguous cleanup strategy that anybody reading the code can understand.