{"id":990,"date":"2014-03-18T13:42:49","date_gmt":"2014-03-18T17:42:49","guid":{"rendered":"https:\/\/blogs.mathworks.com\/steve\/?p=990"},"modified":"2019-11-01T11:00:33","modified_gmt":"2019-11-01T15:00:33","slug":"roifill-design-critique","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/steve\/2014\/03\/18\/roifill-design-critique\/","title":{"rendered":"roifill design critique"},"content":{"rendered":"\r\n

\"Is there anything else you would like to tell us?\"<\/i><\/p>

This was the question whose answer led me to revisit an old friend of mine, the roifill<\/tt> function. This is a function that smoothly fills a specified region of interest in a gray-scale image. I implemented it in the early 1990s following a hallway conversation with another MathWorks developer, who said to me something along the lines of \"I bet you could fill an image region based on the equation for a soap film surface along a wire rim.\"<\/p>

(I don't know when the term inpainting<\/i> became common, but I certainly had not heard it way back then.)<\/p>

Here's an example of roifill<\/tt> in action.<\/p>

I = imread('rice.png'<\/span>);\r\nimshow(I)\r\n<\/pre>\"\" 
If you now call roifill<\/tt>, you can select a polyline around any object you like, and the function will remove the object and smoothly fill in the resulting hole. Here's a sample output (can you find the missing grain of rice?):<\/p>
\"\" <\/p>
Anyway, fast-forward to just two weeks ago. A customer that I know was helping us with a usability test for a planned MATLAB feature. I was sitting in the observation room. After the test tasks were completed, the user experience expert asked a common question: Is there anything else you'd like to tell us about?<\/i><\/p>
It turns out that this customer knows a great deal about the Image Processing Toolbox, and he had some interesting feedback about the roifill<\/tt> function and the mask<\/i> image.<\/p>
roifill<\/tt>, you see, takes a \"mask\" image that specifies the set of pixel values that are to be filled. At least, that's the way I should have designed it. Unfortunately, I didn't, and my design decision was causing a lot of confusion. That's because roifill<\/tt> actually only replaces the interior pixels of the mask.<\/p>
Let me show you a diagram to explain what I mean. Suppose your fill mask looked like this:<\/p>
\"\" <\/p>
It would be a very reasonable expectation to think that roifill<\/i> would replace all the pixels corresponding to this mask. And that's what our intrepid customer thought should happen. But it actually replaces only the interior<\/i> pixels of the mask. That is, it replaces the pixels marked in gray:<\/p>
\"\" <\/p>
The pixels on the edge of the mask are used to establish boundary conditions for the fill equation. These pixels stay the same. Hence the confusion.<\/p>
As I walked back to my office after the usability test, I tried to figure out why I would have originally designed the function this way. I think there were two reasons. First, I was much less experienced with designing image processing functions then. Second, I'm pretty sure I was only thinking about an interactive workflow<\/i>. That is, a user interactively draws a region around an object using a mouse, and then roifill<\/tt> processes the result.<\/p>
In the interactive workflow, it doesn't matter (very much) how the mask border pixels are handled. However, in a noninteractive workflow, I now think it really only makes sense for the mask to represent the set of \"bad pixels\" that all need to be replaced by the function.<\/p>
Ah well. Lessons learned. I will suggest that the Image Processing Toolbox development team think about how to address this behavior quirk in a future release.<\/p>
In the meantime, dear reader ... Is there anything else you would like to tell us?<\/i><\/p>
Let us know by leaving a comment.<\/p>