High Performance File I/O

Today I'd like to tell you how you might improve the performance of reading and writing files using MATLAB's low-level file i/o functions. The key here is to disable the automated buffer flushing.

Experience at MathWorks

The default behavior for fprintf and fwrite is to flush the file buffer after each call to either of these functions. This can have a noticeable performance impact if you are calling either of these functions many times to write small chunks of data each time.

MATLAB has a way to disable the auto-flushing, which is to fopen the file using mode 'W' instead of 'w' (or 'A' instead of 'a' for append). Since fclose also flushes the file, if you have a function that fopens a file, writes small chunks of data to it many times, and then fcloses it, you might want to consider changing your fopen mode.

We made made this change inside wk1write.m recently. When writing out a 200x200 array of doubles using wk1write, the elapsed time shrank from 6.55 seconds down to 4.34 seconds, a savings of 50%.

This technique won't work well if you are in the situation where you need to both read and write to the same file often throughout processing. For that, you need to flush the buffer after each call.

What sorts of file reading/writing bottlenecks do you encounter? Will this tip help you? Let me know.