{"id":14188,"date":"2021-09-03T17:00:21","date_gmt":"2021-09-03T21:00:21","guid":{"rendered":"https:\/\/blogs.mathworks.com\/pick\/?p=14188"},"modified":"2021-09-07T09:11:23","modified_gmt":"2021-09-07T13:11:23","slug":"processing-big-large-blocked-images-blockwise","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/pick\/2021\/09\/03\/processing-big-large-blocked-images-blockwise\/","title":{"rendered":"Processing big\/large\/blocked images, blockwise"},"content":{"rendered":"<div class=\"content\"><!--introduction--><p>Back in R2016b, we introduced <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/tall-arrays.html\">tall arrays<\/a> to facilitate, among other things, processing arbitrarily large datasets. This works nicely for tables or timetables, for example, and works in conjunction with <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/datastore.html\">datastores<\/a>-- repositories for collections of data that are too large to fit in memory.<\/p><p>I'd like to use my POTW pulpit this week to highlight a similar new capability in the image processing domain. In R2021a, we released a \"blockedImage object.\" A blockedImage is an image made from discrete blocks. As in the case of <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/tall.tall.html\"><tt>tall()<\/tt><\/a>, <a href=\"https:\/\/www.mathworks.com\/help\/images\/ref\/blockedimage.html\"><tt>blockedImage()<\/tt><\/a> facilitates processing images or volumes too large to fit into memory. With a blocked image, you can perform processing of arbitrarily large images, in conjunction with <a href=\"https:\/\/www.mathworks.com\/help\/images\/ref\/blockedimagedatastore.html\"><tt>blockedImageDatastore()<\/tt><\/a> without running out of memory.<\/p><!--\/introduction--><h3>Contents<\/h3><div><ul><li><a href=\"#5f721342-57c3-4e22-9d4e-2486c92c37c2\">What about <tt>blockproc()<\/tt>, you ask?<\/a><\/li><li><a href=\"#0da74fe7-ba7b-4807-a9d1-6dee1f6b0081\">You can call 'em \"big,\" you can call 'em \"large,\" or you can call 'em \"blocked\"...<\/a><\/li><\/ul><\/div><h4>What about <tt>blockproc()<\/tt>, you ask?<a name=\"5f721342-57c3-4e22-9d4e-2486c92c37c2\"><\/a><\/h4><p>Sure, we've had blockproc() to facilitate blockwise processing of images since 2009. So what's the big deal?<\/p><p><tt>blockedImage()<\/tt> has several notable advantages over <tt>blockproc()<\/tt>:<\/p><div><ol><li>blockedImage has native support for multiresolution datasets. You can easily make correspondences between image regions in a pyramidal dataset (e.g., \"Where is this feature at the finest resolution level?\") We can answer that because we have support for real-world spatial units.<\/li><li>Conditional processing is much easier with blockedImage. While blockproc works on every block no matter what, blockedImage can skip blocks by using the \"BlockLocationSet\" parameter of blockedImageDatastore. This can be really useful for class balancing during training, for example.<\/li><li>You can specify blocks with blockedImage that don't strictly partition the data. You can have overlapping blocks. You can have blocks with gaps between them.<\/li><li>It's more natural to work with non-image results, such as computing a histogram for each block.<\/li><li>It's a lot easier to work with 3D--and ND!--data.<\/li><li>blockedImage's adapters are much nicer to work with... or not. They're totally optional!<\/li><li>Also the apply() syntax is nicer to work with in blockedImage<\/li><li>In conjunction with blockeImageDatastores, blockedImage provides an easy way to prepare datasets for training machine learning models.<\/li><li>You can create arrays of blockedImages, and do batch processing of images, blockwise!<\/li><li>A \"crop\" method of blockedImage lets you create <i>virtual<\/i> cropped subimages--with no data copy. This is memory efficient, and can be useful if, for example, your image contains multiple logical resolutions or entities--like multiple tissue samples on a single slide image.<\/li><\/ol><\/div><h4>You can call 'em \"big,\" you can call 'em \"large,\" or you can call 'em \"blocked\"...<a name=\"0da74fe7-ba7b-4807-a9d1-6dee1f6b0081\"><\/a><\/h4><p>However you choose to refer to them, processing images too large to fit in memory has never been easier, more powerful, or more flexible! Your legacy code that uses <tt>blockproc()<\/tt> will continue to work, but for your newer analyses--especially if you are training deep learning models--I encourage you to give <tt>blockedImage()<\/tt> a try!<\/p><p>As always, I welcome your <a href=\"http:\/\/blogs.mathworks.com\/pick\/?p=14188#respond\">thoughts and comments<\/a>.<\/p><script language=\"JavaScript\"> <!-- \r\n    function grabCode_b8cc18e16e6646b1b08fafb570bdd4fa() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='b8cc18e16e6646b1b08fafb570bdd4fa ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' b8cc18e16e6646b1b08fafb570bdd4fa';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2021 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_b8cc18e16e6646b1b08fafb570bdd4fa()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n      the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; R2021a<br><\/p><\/div><!--\r\nb8cc18e16e6646b1b08fafb570bdd4fa ##### SOURCE BEGIN #####\r\n%% Processing big\/large\/blocked images, blockwise\r\n%\r\n% Back in R2016b, we introduced\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/tall-arrays.html tall arrays> to\r\n% facilitate, among other things, processing arbitrarily large datasets.\r\n% This works nicely for tables or timetables, for example, and works in\r\n% conjunction with <https:\/\/www.mathworks.com\/help\/matlab\/ref\/datastore.html datastores>REPLACE_WITH_DASH_DASH \r\n% repositories for collections of data that are too large to fit in memory.\r\n%\r\n% I'd like to use my POTW pulpit this week to highlight a similar new\r\n% capability in the image processing domain. In R2021a, we released a\r\n% \"blockedImage object.\" A blockedImage is an image made from discrete\r\n% blocks. As in the case of\r\n% <https:\/\/www.mathworks.com\/help\/matlab\/ref\/tall.tall.html |tall()|>,\r\n% <https:\/\/www.mathworks.com\/help\/images\/ref\/blockedimage.html |blockedImage()|>  \r\n% facilitates processing images or volumes too large to\r\n% fit into memory. With a blocked image, you can perform processing of\r\n% arbitrarily large images, in conjunction with \r\n% <https:\/\/www.mathworks.com\/help\/images\/ref\/blockedimagedatastore.html |blockedImageDatastore()|> \r\n% without running out of memory.\r\n\r\n%% What about |blockproc()|, you ask?\r\n% Sure, we've had blockproc() to facilitate blockwise processing of images since 2009. So what's the big deal?\r\n%\r\n% |blockedImage()| has several notable advantages over |blockproc()|:\r\n% \r\n% \r\n% # blockedImage has native support for multiresolution datasets. You can\r\n% easily make correspondences between image regions in a pyramidal dataset\r\n% (e.g., \"Where is this feature at the finest resolution level?\") We can\r\n% answer that because we have support for real-world spatial units.\r\n% # Conditional processing is much easier with blockedImage. While\r\n% blockproc works on every block no matter what, blockedImage can skip\r\n% blocks by using the \"BlockLocationSet\" parameter of\r\n% blockedImageDatastore. This can be really useful for class balancing during\r\n% training, for example.\r\n% # blockedImage's edge behavior is smarter than blockproc's. blockedImage\r\n% grabs actual pixel values from neighboring blocks; blockproc doesn't.\r\n% # You can specify blocks with blockedImage that don't strictly partition\r\n% the data. You can have overlapping blocks. You can have blocks with gaps\r\n% between them.\r\n% # It's more natural to work with non-image results, such as computing a\r\n% histogram for each block.\r\n% # It's a lot easier to work with 3DREPLACE_WITH_DASH_DASHand ND!REPLACE_WITH_DASH_DASHdata. \r\n% # blockedImage's adapters are much nicer to work with... or not. They're\r\n% totally optional! \r\n% # Also the apply() syntax is nicer to work with in blockedImage\r\n% # In conjunction with blockeImageDatastores, blockedImage provides an\r\n% easy way to prepare datasets for training machine learning models.\r\n% # You can create arrays of blockedImages, and do batch processing of images, blockwise!\r\n% # A \"crop\" method of blockedImage lets you create _virtual_ cropped\r\n% subimagesREPLACE_WITH_DASH_DASHwith no data copy. This is memory efficient, and can be useful\r\n% if, for example, your image contains multiple logical resolutions or\r\n% entitiesREPLACE_WITH_DASH_DASHlike multiple tissue samples on a single slide image.\r\n\r\n%% You can call 'em \"big,\" you can call 'em \"large,\" or you can call 'em \"blocked\"...\r\n% However you choose to refer to them, processing images too large to fit\r\n% in memory has never been easier, more powerful, or more flexible! Your\r\n% legacy code that uses |blockproc()| will continue to work, but for your\r\n% newer analysesREPLACE_WITH_DASH_DASHespecially if you are training deep learning modelsREPLACE_WITH_DASH_DASHI\r\n% encourage you to give |blockedImage()| a try!\r\n\r\n%%\r\n% As always, I welcome your\r\n% <http:\/\/blogs.mathworks.com\/pick\/?p=14188#respond thoughts and comments>.\r\n##### SOURCE END ##### b8cc18e16e6646b1b08fafb570bdd4fa\r\n-->","protected":false},"excerpt":{"rendered":"<p>Back in R2016b, we introduced tall arrays to facilitate, among other things, processing arbitrarily large datasets. This works nicely for tables or timetables, for example, and works in conjunction... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/pick\/2021\/09\/03\/processing-big-large-blocked-images-blockwise\/\">read more >><\/a><\/p>","protected":false},"author":34,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[16],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/14188"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/users\/34"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/comments?post=14188"}],"version-history":[{"count":5,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/14188\/revisions"}],"predecessor-version":[{"id":14203,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/14188\/revisions\/14203"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/media?parent=14188"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/categories?post=14188"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/tags?post=14188"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}