{"id":83,"date":"2007-03-22T14:52:40","date_gmt":"2007-03-22T19:52:40","guid":{"rendered":"https:\/\/blogs.mathworks.com\/loren\/2007\/03\/14\/in-place-operations-on-data\/"},"modified":"2007-04-05T05:59:09","modified_gmt":"2007-04-05T10:59:09","slug":"in-place-operations-on-data","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/loren\/2007\/03\/22\/in-place-operations-on-data\/","title":{"rendered":"In-place Operations on Data"},"content":{"rendered":"<div xmlns:mwsh=\"https:\/\/www.mathworks.com\/namespace\/mcode\/v1\/syntaxhighlight.dtd\" class=\"content\">\r\n   <introduction>\r\n      <p>Recently I mentioned that I would describe a relatively new optimization we have for MATLAB code that allows some function\r\n         calls to operate on variables without allocating more memory for the result.  This can be very beneficial when processing\r\n         large datasets.  I will describe and demonstrate this feature using R2007a.\r\n      <\/p>\r\n   <\/introduction>\r\n   <h3>Contents<\/h3>\r\n   <div>\r\n      <ul>\r\n         <li><a href=\"#1\">Make Data<\/a><\/li>\r\n         <li><a href=\"#8\">Required Characteristics of In-place Behavior<\/a><\/li>\r\n         <li><a href=\"#13\">Considerations<\/a><\/li>\r\n         <li><a href=\"#14\">Limitations<\/a><\/li>\r\n         <li><a href=\"#15\">Feedback?<\/a><\/li>\r\n      <\/ul>\r\n   <\/div>\r\n   <h3>Make Data<a name=\"1\"><\/a><\/h3>\r\n   <p>Create a \"large\" dataset.  This one works for me with 1GB RAM and 1.5GB of swap space running Windows XP and no other applications.<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">n = 38*2^20;\r\nx = randn(n,1);<\/pre><p>Now call another function that will call either a regular function or an in-place function.<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">inplaceTest(x)<\/pre><p>Let's look at the code for <tt>inplaceTest<\/tt>.\r\n   <\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">type <span style=\"color: #A020F0\">inplaceTest<\/span><\/pre><pre style=\"font-style:oblique\">\r\nfunction inplaceTest(x)\r\n% Call functions with either regular or in-place semantics.\r\n\r\n%% Call a Regular Function with the Same Left-Hand Side\r\nx = myfunc(x); \r\n%% Call an In-place Function with the Same Left-Hand Side\r\nx = myfuncIP(x); \r\n%% Call a Regular Function with a Different Left-Hand Side\r\ny = myfunc(x); \r\n%% Call an In-place Function with Same Left-Hand Side\r\n% Note: if we changed this next call to assign output to a new LHS, we get an error\r\nx = myfuncIP(x); \r\n<\/pre><p>It just gets the data in and twice calls two different functions.<\/p>\r\n   <p>See what the Windows Task Manager shows for a recent run of mine for the 4 calculations.  You can see when the CPU was busy\r\n      and you can match that with the times that memory was allocated, whether temporary or permanent. Here's how a typical run\r\n      of this code appears in the Windows Task Manager Performance tab when I run it.\r\n   <\/p>\r\n   <p><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/loren\/83\/taskManager.png\"> <\/p>\r\n   <p>You can see the four calls to the computational functions in the CPU Usage History, and you can see the amount of extra memory\r\n      used in the Page File Usage History.  When we call the in-place function using the same input and output variable names from\r\n      the test function, no extra memory is allocated.\r\n   <\/p>\r\n   <h3>Required Characteristics of In-place Behavior<a name=\"8\"><\/a><\/h3>\r\n   <p>Let's look at the code for the two functions <tt>myfunc<\/tt> and <tt>myfuncIP<\/tt> so we can see how they differ.\r\n   <\/p>\r\n   <p>Here's the function <tt>myfunc.m<\/tt><\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">type <span style=\"color: #A020F0\">myfunc<\/span><\/pre><pre style=\"font-style:oblique\">\r\nfunction y = myfunc(x)\r\ny = sin(2*x.^2+3*x+4);\r\n<\/pre><p>and here's the function <tt>myfuncIP.m<\/tt><\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">type <span style=\"color: #A020F0\">myfuncIP<\/span><\/pre><pre style=\"font-style:oblique\">\r\nfunction x = myfuncIP(x)\r\nx = sin(2*x.^2+3*x+4);\r\n<\/pre><p>As you can see, the two <i><tt>myfunc<\/tt><\/i> functions differ. The in-place version has a return argument <i>with the same name<\/i> as one of the input arguments, in this case, <tt>x<\/tt>.  Also, the computation needs to put its results into <tt>x<\/tt> instead of <tt>y<\/tt>, since the new return variable name is now different.\r\n   <\/p>\r\n   <p>The second important idea required to take advantage of in-place computation is that the in-place function must itself be\r\n      called from another function, as I've done with <tt>inplaceTest<\/tt>.\r\n   <\/p>\r\n   <h3>Considerations<a name=\"13\"><\/a><\/h3>\r\n   <p>I'd like to pass on several tips and rules of thumb about in-place functions.  This is not an ordered list.<\/p>\r\n   <div>\r\n      <ul>\r\n         <li>Because of MATLAB's JIT, some functions or parts of code are already doing some work in place.  In these cases, you will see\r\n            less or no benefit than you might expect by switching to in-place operations.  For example, using the binary version of <tt>max<\/tt> (e.g, <tt>A=max(A,B)<\/tt>) doesn't gain, even if in a function.\r\n         <\/li>\r\n         <li>Benefits don't usually occur for the smallest arrays.  It takes typically at least 1000 elements before you might see a difference.<\/li>\r\n         <li>Keep your code natural.  Don't write unnatural code just to take advantage of this.  This is particularly true if your function\r\n            is already computationally intensive. There is more speed gain for simpler functions, though memory might still be a concern.\r\n             For example, <tt>lu<\/tt> is an order <tt>n^3<\/tt> operation, and the memory speed gain is only order <tt>n^2<\/tt>.  So it's not worth doing contortions to write <tt>[L A P] = lu(A)<\/tt> (more naturally written <tt>[L U P] = lu(A)<\/tt>), unless you really need to conserve memory.\r\n         <\/li>\r\n      <\/ul>\r\n   <\/div>\r\n   <h3>Limitations<a name=\"14\"><\/a><\/h3>\r\n   <p>There are some limitations regarding when the in-place optimization is operative.  These include:<\/p>\r\n   <div>\r\n      <ul>\r\n         <li>Not all built-in MATLAB functions currently obey in-place semantics. We tried to support the most important and obvious ones\r\n            first such as elementwise operations.  There is not a list of these functions.  It will grow from release to release.\r\n         <\/li>\r\n         <li>There is no interface for the in-place operations via MEX-files.<\/li>\r\n         <li>MATLAB should be able to recognize in-place possibilities even when variable names don't match between the input and output\r\n            names.\r\n         <\/li>\r\n      <\/ul>\r\n   <\/div>\r\n   <h3>Feedback?<a name=\"15\"><\/a><\/h3>\r\n   <p>Do you plan to take advantage of this new feature in your code?  Let me know <a href=\"https:\/\/blogs.mathworks.com\/loren\/?p=83#respond\">here<\/a>.\r\n   <\/p><script language=\"JavaScript\">\r\n<!--\r\n\r\n    function grabCode_e726c956a43643db9bd78a65b07314f6() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='e726c956a43643db9bd78a65b07314f6 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' e726c956a43643db9bd78a65b07314f6';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        author = 'Loren Shure';\r\n        copyright = 'Copyright 2007 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add author and copyright lines at the bottom if specified.\r\n        if ((author.length > 0) || (copyright.length > 0)) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (author.length > 0) {\r\n                d.writeln('% _' + author + '_');\r\n            }\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n      \r\n      d.title = title + ' (MATLAB code)';\r\n      d.close();\r\n      }   \r\n      \r\n-->\r\n<\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_e726c956a43643db9bd78a65b07314f6()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n            the MATLAB code \r\n            <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; 7.4<br><\/p>\r\n<\/div>\r\n<!--\r\ne726c956a43643db9bd78a65b07314f6 ##### SOURCE BEGIN #####\r\n%% In-place Operations on Data \r\n% Recently I mentioned that I would describe a relatively new optimization\r\n% we have for MATLAB code that allows some function calls to operate on\r\n% variables without allocating more memory for the result.  This can be very\r\n% beneficial when processing large datasets.  I will describe\r\n% and demonstrate this feature using R2007a.\r\n\r\n%% Make Data\r\n% Create a \"large\" dataset.  This one works for me with 1GB RAM and 1.5GB\r\n% of swap space running Windows XP and no other applications.\r\nn = 38*2^20;\r\nx = randn(n,1);\r\n\r\n%%\r\n% Now call another function that will call either a regular function or an\r\n% in-place function.\r\ninplaceTest(x)\r\n%%\r\n% Let's look at the code for |inplaceTest|.\r\ntype inplaceTest\r\n%%\r\n% It just gets the data in and twice calls two different functions.\r\n%%\r\n% See what the Windows Task Manager shows for a recent run of mine for the\r\n% 4 calculations.  You can see when the CPU was busy and you can match\r\n% that with the times that memory was allocated, whether temporary or\r\n% permanent.\r\n% Here's how a typical run of this code appears in the Windows Task\r\n% Manager Performance tab when I run it.\r\n%%\r\n% <<taskManager.png>>\r\n%%\r\n% You can see the four calls to the computational functions in the CPU\r\n% Usage History, and you can see the amount of extra memory used in the\r\n% Page File Usage History.  When we call the in-place function using the\r\n% same input and output variable names from the test function, no extra\r\n% memory is allocated.\r\n%% Required Characteristics of In-place Behavior\r\n% Let's look at the code for the two functions |myfunc| and |myfuncIP| so\r\n% we can see how they differ.\r\n%%\r\n% Here's the function |myfunc.m|\r\ntype myfunc\r\n%%\r\n% and here's the function |myfuncIP.m|\r\ntype myfuncIP\r\n%%\r\n% As you can see, the two _|myfunc|_ functions differ.\r\n% The in-place version has a return argument _with the same name_ as\r\n% one of the input arguments, in this case, |x|.  Also, the computation\r\n% needs to put its results into |x| instead of |y|, since the new return \r\n% variable name is now different.\r\n%%\r\n% The second important idea required to take advantage of in-place\r\n% computation is that the in-place function must itself be called from\r\n% another function, as I've done with |inplaceTest|.\r\n%% Considerations\r\n% I'd like to pass on several tips and rules of thumb about in-place\r\n% functions.  This is not an ordered list.\r\n%\r\n% * Because of MATLAB's JIT, some functions or parts of code are already\r\n% doing some work in place.  In these cases, you will see less or no\r\n% benefit than you might expect by switching to in-place operations.  For\r\n% example, using the binary version of |max| (e.g, |A=max(A,B)|) doesn't\r\n% gain, even if in a function. \r\n% * Benefits don't usually occur for the\r\n% smallest arrays.  It takes typically at least 1000 elements before you\r\n% might see a difference. \r\n% * Keep your code natural.  Don't write unnatural\r\n% code just to take advantage of this.  This is particularly true if your\r\n% function is already computationally intensive. There is more speed gain\r\n% for simpler functions, though memory might still be a concern.  For\r\n% example, |lu| is an order |n^3| operation, and the memory speed gain is\r\n% only order |n^2|.  So it's not worth doing contortions to write |[L A P] =\r\n% lu(A)| (more naturally written |[L U P] = lu(A)|), unless you really need to conserve memory.\r\n%% Limitations\r\n% There are some limitations regarding when the in-place optimization is\r\n% operative.  These include:\r\n% \r\n% * Not all built-in MATLAB functions currently obey in-place semantics. We\r\n% tried to support the most important and obvious ones first such as\r\n% elementwise operations.  There is not a list of these functions.  It will\r\n% grow from release to release.\r\n% * There is no interface for the in-place operations via MEX-files.\r\n% * MATLAB should be able to recognize in-place possibilities even when\r\n% variable names don't match between the input and output names.\r\n%% Feedback?\r\n% Do you plan to take advantage of this new feature in your code?  Let me\r\n% know <https:\/\/blogs.mathworks.com\/loren\/?p=83#respond here>. \r\n\r\n\r\n##### SOURCE END ##### e726c956a43643db9bd78a65b07314f6\r\n-->","protected":false},"excerpt":{"rendered":"<p>\r\n   \r\n      Recently I mentioned that I would describe a relatively new optimization we have for MATLAB code that allows some function\r\n         calls to operate on variables without allocating more... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/loren\/2007\/03\/22\/in-place-operations-on-data\/\">read more >><\/a><\/p>","protected":false},"author":39,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[16,10,7,6],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/83"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/comments?post=83"}],"version-history":[{"count":0,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/posts\/83\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/media?parent=83"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/categories?post=83"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/loren\/wp-json\/wp\/v2\/tags?post=83"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}