{"id":34,"date":"2006-04-19T14:23:54","date_gmt":"2006-04-19T19:23:54","guid":{"rendered":"https:\/\/blogs.mathworks.com\/loren\/?p=34"},"modified":"2016-07-28T14:28:45","modified_gmt":"2016-07-28T19:28:45","slug":"high-performance-file-io","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/loren\/2006\/04\/19\/high-performance-file-io\/","title":{"rendered":"High Performance File I\/O"},"content":{"rendered":"<p>Today I'd like to tell you how you might improve the performance of reading and writing files using MATLAB's low-level file i\/o functions.  The key here is to disable the automated buffer flushing.\r\n<\/p>\r\n<h3>Experience at MathWorks<\/h3>\r\n<p>\r\nThe default behavior for <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/fprintf.html\"><kbd>fprintf<\/kbd><\/a> and <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/fwrite.html\"><kbd>fwrite<\/kbd><\/a> is to flush the file buffer after each call to either of these functions. This can have a noticeable performance impact if you are calling either of these functions many times to write small chunks of data each time.\r\n<\/p>\r\n<p>\r\nMATLAB has a way to disable the auto-flushing, which is to <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/fopen.html\"><kbd>fopen<\/kbd><\/a> the file using mode <kbd>'W'<\/kbd> instead of <kbd>'w'<\/kbd> (or <kbd>'A'<\/kbd> instead of <kbd>'a'<\/kbd> for append). Since <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/fclose.html\"><kbd>fclose<\/kbd><\/a> also flushes the file, if you have a function that <kbd>fopen<\/kbd>s a file, writes small chunks of data to it many times, and then <kbd>fclose<\/kbd>s it, you might want to consider changing your <kbd>fopen<\/kbd> mode.\r\n<\/p>\r\n<p>\r\nWe made made this change  inside <kbd>wk1write.m<\/kbd> recently. When writing out a 200x200 array of doubles using <kbd>wk1write<\/kbd>, the elapsed time shrank from 6.55 seconds down to 4.34 seconds, a savings of 50%.\r\n<\/p>\r\n<p>\r\nThis technique won't work well if you are in the situation where you need to both read and write to the same file often throughout processing.  For that, you need to flush the buffer after each call.\r\n<\/p>\r\n<p>\r\nWhat sorts of file reading\/writing bottlenecks do you encounter?  Will this tip help you?  <a href=\"?p=34#respond\">Let me know<\/a>.\r\n<\/p>","protected":false},"excerpt":{"rendered":"<p>Today I'd like to tell you how you might improve the performance of reading and writing files using MATLAB's low-level file i\/o functions.  The key here is to disable the automated buffer... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/loren\/2006\/04\/19\/high-performance-file-io\/\">read more >><\/a><\/p>","protected":false},"author":39,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[10],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/34"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/comments?post=34"}],"version-history":[{"count":2,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/34\/revisions"}],"predecessor-version":[{"id":1800,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/34\/revisions\/1800"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/media?parent=34"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/categories?post=34"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/tags?post=34"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}