Loren on the Art of MATLAB

Keeping Things Tidy 28

Posted by Loren Shure,

In the past, when you opened a file in a MATLAB program, you had to keep track of it so you could close it under all possible conditions, i.e., if reading or processing failed, worked partially, or worked perfectly. You could achieve this using the catch block after a try. But you still might have to know before closing the file if everything you wanted to do was complete. Code could easily get awkward and buggy if there were lots of code paths to watch out for. With R2008a, you can use the onCleanup class to help you out more cleanly.

Contents

Example Code - Open File

Here's the start of a file I've written that opens up an image file for reading, using fopen.

type openImageFile0
function openImageFile0
% openImageFile Open an image file.
fd = fopen('pout.tif');

If I run it, you can see that I never close the file.

openImageFile0()
fids = fopen('all')
filename = fopen(fids(1));
filename(end-30:end)
fids =
     3
ans =
toolbox\images\imdemos\pout.tif

Tidy up now.

fclose(fids(1));

Example Code - Manage Open, Close, and Errors Using try/catch

Now let's look at another version of the code that will close the file it opens before returning.

type openImageFile1
function openImageFile1
% openImageFile1 Open/process file while ensuring file is 
% closed on finish.
try
    fd = fopen('pout.tif');
    if fid == -1
        error('LS:NoImageFile','no such file')
    end
    doMoreHere(fd);
catch Ex
    disp('Can not read file')
end
if fid ~= -1
    fclose(fid);
end

   

You can see that, even with try/catch, since I always want to close the image file, I have some extra logic at the end of the file. And I have to be careful to know whether the error occurs because the file was never opened, or if instead the error occurs in later processing.

Example Code - Use onCleanup to Ensure File Closes

Here's a file roughly equivalent to the first one, but it ensures that the image file is closed when the function is finished running.

type openImageFile2
function openImageFile2
% openImageFile2 Open an image file for further processing.
%   Note: this doesn't do what you want, since the file will
%   be closed after completing and the file identifier won't
%   be active for use upon completion.
fd = fopen('pout.tif');
C = onCleanup(@()fclose(fd));

To see that this works we first check that no files are open, run the function, and make sure that no files are open afterwards.

fb = fopen('all');
openImageFile2()
fp = fopen('all');
noFilesOpen = isequal(fp,fb,[])
noFilesOpen =
     1

Example Code - Use onCleanup and Process After Opening the File

Let's see the next version of the file.

type openImageFile3
function fd = openImageFile3
% openImageFile3 Open an image file for further processing.
%   Note: this doesn't do what you want, since the file will 
%   be closed after completing and the file identifier won't
%   be active for use upon completion.
fd = fopen('pout.tif');
C = onCleanup(@()fclose(fd));

This function returns the file identifier, but as we'll see, the file is closed once the function returns. This means that any processing/reading we were planning to do with the file after calling the open function will not work (since the file will no longer be open).

fb = fp;
fid = openImageFile3()
fp = fopen('all');
noFilesOpen = isequal(fp,fb,[])
fid =
     3
noFilesOpen =
     1

Even though fid is returned from the function call, the file is no longer open. This is because the onCleanup object C is cleared from existence once openImageFile3 completes execution.

Example Code - Use onCleanup and Process and Close File on Completion

Here's the last version of the file for this blog. As long as you call it with 2 outputs, you are good to go.

type openImageFile4
function [fd, C] = openImageFile4
% openImageFile4 Open an image file for further processing.
%   Note: this does what you want, provided you call the 
%   function with both output arguments.  When you are done
%   cleaned up and the onCleanup object will trigger the file
%   closing.

% Don't even open file if function isn't called with 2 outputs.
if nargout < 2
    error('LS:MustHave2Outputs','must call with 2 outputs')
end

fd = fopen('pout.tif');
C = onCleanup(@()fclose(fd));

This version returns the file identifier and the onCleanup object. Once you have processed your file, simply clear the onCleanup object and the file will close.

To see how this might work, we call it from our function that does the processing.

type run4
function y = run4
% run4 processes an image file.
[fid,C] = openImageFile4();
blocksize = 100;
y = [];
while something
    dat = fread(fid,count);
    y = doSomething(y,dat);
    something = update(something);
end

You can see that run4 calls openImageFile4, and then does the required processing. Because C will vanish once run4 completes, the file will be closed.

Other Uses?

The onCleanup object allows you to more cleanly manage resources such as closing files, closing windows (e.g., last week someone mentioned managing instances of waitbar), managing switching directories. What uses do you foresee for the onCleanup object in your work? Let me know here.


Get the MATLAB code

Published with MATLAB® 7.6

28 CommentsOldest to Newest

Maybe this will grow on me, but I must admit, it seems somehow sloppy to set things up so that a job like closing the file is called implicitly by having an object disappear because the function whose scope it belongs to ends. Also, as I’m looking back through this blog, how does this help us deal with the possibilities of errors cropping up? Suppose that either 1. pout.tif doesn’t exist, or 2. you run into a problem in trying to use it. Does the onCleanup somehow save us from throwing an exception in your last example?

Thanks,
Dan

I’ve got several comments on this topic.

First, I can see a general benefit to this type of function since it protects from a general type of error. Although, part of me is wondering why the fundamental design of the MatLab language doesn’t provide adequate protection. Maybe Dan’s comment is related to this issue.

Does this command require the “in-line” generation of the function handle? Or can you generate the function handle on a previous line. If so, does this eliminate the empty function argument? Again if so, can you modify your openImageFile2 code to illustrate the separate line function handle?

Finally, since function handles are a central issue in this column, you can be sure to generate criticism from me regarding MatLab’s documentation on function handles. The FRP for such a complex topic needs dozens of examples, not just two. Further, the FRP ignores the topic of an argument list for a non-anonymous function. Finally, why do the smart folks at The MathWorks think it is useful that MatLab, “. . . provides a means of calling a function indirectly.” It seems that sound programming technique would be to avoid anything that is indirect.

Dan-

In the last example, there is not enough error checking yet so an exception can still be thrown (I was trying to illustrate something else though, about triggering the close of the file).

–Loren

Oliver-

You can generate the function handle however you want. It simply must be one that requires no input arguments. The reason a regular function handle isn’t mentioned on the FRP is because of requiring that all input arguments are dealt with to the function being called.

–Loren

It seems strange that (a task like) closing a file that is no longer used needs to be handled explicitly by the programmer at all. In Matlab programs, when data is no longer needed (i.e. when a variable holding a matrix goes out of scope), Matlab reclaims memory automatically. I think this would lead most new Matlab users to assume that the same mechanism would be used for other resources, such as file handles. I don’t see why Matlab couldn’t simply implement this behaviour: when a file handle goes out of scope, the file should automatically be closed.

I realise that changing the behaviour of fopen would break existing programs—not a reasonable solution. I would advocate deprecating fopen and replacing it with a function that returns an object that, when it goes out of scope, automatically closes the file.

The new fopen would require the returned handle to be assigned to a variable (in the way that, I suspect, onCleanup requires one to assign its return value to a variable) so that Matlab can detect when it goes out of scope.

While onCleanup offers a mechanism for programmers to specify their own cleanup behaviour, I think it is unnecessary. If Matlab could automatically reclaim resources, such as by closing files as described above—as it already does with most other built-in data types—cleanup is just object destruction, which is also provided by Matlab.

Interesting thoughts, C Rose. Unfortunately, we have too many people with too much code written to change the way fopen works now. But your point is well taken — about automatic cleanup when we can. I still think onCleanup is useful for times when you want figure windows to close, for example. There’s no guaranteed way to know the user is done with them, unless the application itself knows.

–Loren

Interesting idea, but I have to agree with the sentiments above that this isn’t the cleanset of solutions. The auto-cleaning file handle would be a great addition, though.

Anyways, when I read the post I thought about my mex files that I quite frequently use and which frequently remain in memory untill the next ‘clear all’, consuming large ammounts of memory. Is there a way to use this onCleanup trick to unload a mex file?

–DA

Daniel-

The unloading of mex would have to me from something other than the mex-file itself, I think. If it was called from a MATLAB M-file, you could do something like:

function fred
C = oncleanup(@()clear('callMex')
callMex

When the call to fred finishes, the mex-file callMex will be cleared.

–Loren

As I read the Matlab documentation the implementation of onCleanup is fatally flawed and I don’t think anybody should use it unless some things can be clarified.

According to

help for onCleanup

‘the lifecycle of an object ends *sometime* between the time its variable is no longer used inside the function and the end of the function’

The implementation of onCleanup seems to rely on a subtly different interpretation of object lifecyle.

‘the lifecycle of an object ends at the end of the function within which it’s last reference is lost’

This means the following example

function y = run4
% run4 processes an image file.
[fid,C] = openImageFile4();
blocksize = 100;
y = [];
while something
…dat = fread(fid,count);
…y = doSomething(y,dat);
…something = update(something);
end

is broken.

According to Matlab spec C could be deleted any time after it appears in the line

[fid,C] = openImageFile4();

thus deleting the file handle whilst it was being used. It looks like a classic race condition problem.

To be safe you could rewrite the code

function y = run4
% run4 processes an image file.
[fid,C] = openImageFile4();
blocksize = 100;
y = [];
while something
…dat = fread(fid,count);
…y = doSomething(y,dat);
…something = update(something);
end
C;

But somehow I see the Matlab optimiser at some time in the future stripping out such dead code and rendering the trick useless anyway.

Stylistically it is also not obvious to a reader of the code what C is being used for and it’s inclusion in the code could easily be deleted by an eager maintainer unless the ode is accompanied by copious documentation.

There are however neater ways to solve the same problem in other languages.

The Python way:

http://docs.python.org/whatsnew/pep-343.html

with open(‘/etc/passwd’, ‘r’) as f:
….for line in f:
……..print line

lexical scoping is used to clarify the required lifetime of the file handle f. As well if any exception is thrown from within the block the file is also closed.

The Ruby way:

File.open ‘/etc/passwd’ do |f|
…f.each_line do |l|
……puts l
…end
end

Very similar to the python way and again there is no requirement by the user to provide any cleanup or close functionality. In case of normal block termination or in the event of an error the open function will *always* close the file handle when the block terminates.

The onCleanup functionality could also be dealt with by adding an ensure keyword to the Matlab language as has Ruby and Python.

function do_open(file, flags, action)
… fid = fopen(file, flags)
… try
…….action(fid)
….ensure
…….fclose(fid)
….end
end

Notice there is no catch above. We don’t want to catch an error we just want to *ensure* that we cleanup if there is an error or not. Any error is propagated to code further upstream to handle. The ensure can be substituted with a catch and rethrow but is a bit uglier as you need to write code to handle closing the file if and if not an error occurs.

Using do_open in Matlab would be like

do_open(‘/etc/passwd’, ‘r’, @body)
function body(fid)
… read(fid, 10)
… function_with_error()
end

As is obvious the function definition is kind of superfluous and would be nicer with some kind of multi line anonymous function

do_open(‘/etc/passwd’, ‘r’ ) @(fid)
… read(fid, 10)
… function_with_error()
end

Brad Phelan

Loren, you say that you can’t change the way fopen works because that would break too much existing code. MATLAB has changed a lot over the years, breaking large chunks of my code on occasion. It’s very difficult trying to keep code compatible with older versions of MATLAB. For example, LOGIC arrays no longer being numeric was a *huge* change. Compared to that, changing the file pointers from a scalar integer value to an object seems rather trivial. Most code would still work the way it does now, especially if you can test for validity of the file pointer by examining its the numeric value (i.e. f>0).

Here’s one more vote for the self-closing file handle!

Daniel:
You can unload your MEX-files with ‘clear functions’.

Brad,

onCleanup objects obey the same rules as MATLAB objects do, since onCleanup is implemented as a MATLAB object. It actually has a rather trivial implementation, which you can see by looking at onCleanup.m.

I think you stated it very well when you said, “The lifecycle of an object ends at the end of the function within which its last reference is lost.” This is mostly true for MATLAB objects, including onCleanup objects. Objects may be deleted sooner explicitly with the delete or clear functions, or implicitly by reusing the variable containing the last reference to the object.

For typical uses, onCleanup objects will not be returned as output arguments from the functions which create them. Also, most uses will not involve explicit or implicit early destruction of onCleanup objects. Thus, the lifecycle of the onCleanup object will be exactly the scope of the function which contains it.

This is what Loren’s run4 would look like if the creation of the onCleanup object were not encapsulated in a separate function:

function y = run4
    % run4 processes an image file.
    fid = fopen('pout.tif');
    C = onCleanup(@()fclose(fid));
    blocksize = 100;
    y = [];
    while something
        dat = fread(fid,count);
        y = doSomething(y,dat);
        something = update(something);
    end
end

-=>J

Hi Jason,

From the Matlab Documentation

http://tinyurl.com/2drurc

(Object Lifecycle) Inside a Function.

The lifecycle of an object referenced by a local variable or
input argument is defined to exist from the time the
variable is assigned until the time it is reassigned,
cleared, or no longer referenced within that function or any
handle array. If the variable is not cleared or reassigned,
the lifecycle of an object ends sometime between the time
its variable is no longer used inside the function and the
end of the function.

Within the time between a variable’s
last use and the end of function execution, the MATLAB
runtime destroys the object to which the variable refers

The key point is the last sentence. The Matlab runtime is allowed to delete an object before the end of the function. That is the object may be deleted before the function has finished executing. If this occured ( and it probably doesn’t at the moment) then the oncleanup implementation will fail.

Either the documentation is wrong or the implementation of onCleanup makes an assumption which may not be future proof. Personally I think the documentation makes sense and it gives Matlab future flexibility for gc optimization.

Regards

Brad Phelan

Brad,

It is true that a garbage collection algorithm could delete an unused variable once the last use has passed. There isn’t currently such an algorithm in MATLAB, but if you want to protect against a future implementation of one, you may explicitly delete or clear the onCleanup object, which will define a use of the variable.

This happens to make for rather readable code, actually:

function y = run4
% run4 processes an image file.
fid = fopen(’pout.tif’);
    C = onCleanup(@()fclose(fid));
    blocksize = 100;
    y = [];
    while something
        dat = fread(fid,count);
        y = doSomething(y,dat);
        something = update(something);
    end
    C.delete;
end

-=>J

The issue is not whether it works or not or that it can be made to work through a series of hacks. The problem is that the documentation is inconsistent. Either the documentation on object lifecyle should be changed to insist that objects last until the end of a function block or the implementation or documentation of onCleanup needs fixing to be consistent. Sorry to be picky but this is a feature promoted in a high profile blog and should be guaranteed to work in all future Matlab releases.

I notice that Octave ( the open source Matlab clone ) has a feature called unwind_protect.

http://www.cheric.org/education/eduaids/octave/octman/octave_41.html

unwind_protect
…while true
…end
unwind_protect_cleanup
…fprintf (‘caught interrupt\n’);
end

which behaves the same as ensure in python and ruby. The code inside unwind_protect_cleanup will run regardless of event of an exception in the unwind_protect block. Even a ^C keypress will force the cleanup code to run.

It is also a clear and unambiguous cleanup strategy that anybody reading the code can understand.

Forgive the late comment! I recently came across some cleanup behaviour that is confusing to me, so I thought I’d bring it up here.

I have been using a nested function as my clean up function, mainly for neatness as I want to do a number of things on cleanup. Here is some simplified sample code:

function my_main_function()
c = onCleanup(@clean_up);

% Some code that may error…

% Define p.old_default_stream
% Define epb (using embededprogressbar).

% Some other code that may error.

function clean_up()
% Get rid of the progress bar.
if exist(‘epb’, ‘var’)
delete(epb);
end
% Reset the default stream.
if exist(‘p’, ‘var’) && isfield(p, ‘old_default_stream’)
RandStream.setDefaultStream(p.old_default_stream);
end
end
end

This code works fine as is, but there is an mlint warning ‘The variable ‘epb’ is an uplevel variable, invalid in a function called by onCleanup’ in 2009a. Note that this warning only applies to ‘epb’, not to ‘p’. What does this really mean? The variable does seem to exist in whos in the nested function.

After a bit of playing around I noticed that if I change the onCleanup line to c = onCleanup(@() clean_up); that the mlint error disappears. Does this mean that the problem is fixed, or is it just an mlint effect? Is the difference between the 2 that the first one is a function handle, but the second is an anonymous function? Why does it make a difference in onCleanup?

Do you have any guidance for using nested functions (or other functions) in onCleanup?

Thanks,
Kieran

Kieran-

I can’t explain the mlint message and think it’s probably too strongly worded. It does look like you are trying to work around the existence issue in the nested function. Your code might be cleaner and perhaps mlint would be happy if instead of using exist, you placed statements in try/catch, something like this:

try 
  delete(epb)
catch ME
end
try
  if isfield(p,'old_default_stream')
    Rand....
  end
catch ME
end

–Loren

Hi Kieran,

It’s probably not a good idea to use nested functions for your cleanup routines. The problem is that the variables in the parent function scope are cleared in an unspecified order, and the onCleanup is one of those variables. There is no guarantee that your variables aren’t cleared before the onCleanup. The workaround that you are using is merely tricking M-Lint, it is not solving your problem.

The best way to use onCleanup is to explicitly give the onCleanup references to any variables which it will need to refer.

I recommend creating individual onCleanup objects for your tasks:

p.old_default_stream = RandStream.getDefaultStream;
c1 = onCleanup(@()RandStream.setDefaultStream(p.old_default_stream));

epb = embededprogressbar;
c2 = onCleanup(@()delete(epb));

Note that by doing it this way, you don’t have to check if the variables exist, because you create the onCleanup objects immediately after you create the dependent variables, and the onCleanup maintains it’s own reference to the variable, preventing it from being cleared.

Hope that helps,

-=>J

Jason, Loren,

Thanks for the replies. Now that I understand the unspecified clearance order it all makes sense to me. I was trying to be too general in my code.

Presumably then onCleanup should never be called with a function handle, but always with an anonymous function? If so then maybe use with a function handle should generate an error.

Thanks for your help,
Kieran

Kieran-

A plain function handle could be fine in some cases, I think. In any case, you can rewrite @func as @()func() if it really doesn’t take any input.

–loren

Loren,

I know this is an old thread (exactly 2 years now that I think of it!), but I hope that you are still reading it.

I want to use something like onCleanup when a script m-file is aborted via Ctrl-C. My use case is that I have opened a number of Excel-workbooks as handles and write to them with a modified version of xlswrite (for speed considerations I don’t want to use the regular xlswrite which opens/closes the workbooks every time). I want the workbooks to be closed when the program is aborted, but onCleanup isn’t triggerd because the scripts variables don’t go out of scope. Obviously I could wrap the whole program into a function but that is not really what I want. Can you think of any workarounds?

thank you
Paul

Paul-

MATLAB doesn’t have any facility for handling Ctrl-C at the programming level that I know of. I don’t know why you don’t want to use a function, but that solution would be my preferred one, for this, and probably a bunch of other reasons.

–Loren

Hi Loren,

Is there a way to “cancel” an onCleanup object’s task (i.e., delete the object without triggering its task)?

Ross-

As far as I know, there is no way to change it. There’s an outside chance that there’s a way to change the callback, but I don’t know if that’s possible, but if so, it should be in the documentation. Unfortunately, I have no access to MATLAB at the moment.

–Loren

I am currently programming GUIs using Matlab. I would like to know if there is a way to detect when the user closes a GUI using the “X” in the top bar of the GUI window. I refer to this method of closing the GUI as a “brute force” close as opposed to closing the GUI via code using the close() or delete() commands, for example.

The reason I ask, is that I have a GUI with a pushbutton callback. The callback calls another function (FNCx) with a long execution time. Before calling FNCx, however, the callback launches another GUI (much like a waitbar display). The waitbar is updated from within the FNCx. If the user brute force closes the custom waitbar GUI it causes an error when FNCx attempts to update the waitbar because the handle of the waitbar GUI is no longer valid.

If there were a means to detect a brute force close it would help simplify my code.

Regards,
Ken

@Ken,

You can use “isvalid” to check to see if the handle of waitbar is valid before updating it.

If you want the closing of the waitbar to trigger something else, you can assign a “DeleteFcn” for your waitbar.

is it possible to use onCleanup to close an avifile using the command

aviobj = close(aviobj)

thanks

Flori-

That should work. If you have a problem with it, please contact technical support – link at the right of this blog.

–Loren

These postings are the author's and don't necessarily represent the opinions of MathWorks.