{"id":344,"date":"2010-10-08T12:00:00","date_gmt":"2010-10-08T16:00:00","guid":{"rendered":"https:\/\/blogs.mathworks.com\/steve\/2010\/10\/08\/the-two-amigos\/"},"modified":"2019-10-29T13:40:20","modified_gmt":"2019-10-29T17:40:20","slug":"the-two-amigos","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/steve\/2010\/10\/08\/the-two-amigos\/","title":{"rendered":"The Two Amigos"},"content":{"rendered":"<div xmlns:mwsh=\"https:\/\/www.mathworks.com\/namespace\/mcode\/v1\/syntaxhighlight.dtd\" class=\"content\">\r\n   <introduction>\r\n      <p>I'd like to introduce Brett Shoelson as guest blogger for today. Many of you already know Brett from his work on the <a href=\"https:\/\/blogs.mathworks.com\/pick\/\">File Exchange Pick of the Week blog<\/a>, or his posts on the <a>comp.soft-sys.matlab<\/a> newsgroup, or from his many presentations given on the road. I met Brett \"virtually\" many years ago, long before he came\r\n         to work for MathWorks, through his posts on the newsgroup. Brett's been doing fun, interesting, and even useful things to\r\n         images using MATLAB for quite a long time.\r\n      <\/p>\r\n      <p>A few weeks ago Brett posted the <a href=\"https:\/\/blogs.mathworks.com\/pick\/2010\/09\/03\/two-amigos\/\">Two Amigos Challenge<\/a> on the Pick of the Week blog. Readers of the blog know that long-time contributor Bob Bemis (Hi, Bob!) has moved on from\r\n         MathWorks and isn't posting there anymore. Brett challenged his readers to digitally retouch the \"Three Amigos\" picture on\r\n         the blog and turn it into a \"Two Amigos\" picture using only MathWorks products.\r\n      <\/p>\r\n      <p>I was so impressed with Brett's own solution to the Challenge that I invited him to show it to my readers here.<\/p>\r\n      <p>So here is Brett's solution to the Two Amigos Challenge ...<\/p>\r\n      <p>... in which we remove a subject from a photograph, and digitally retouch it to conceal the modification...<\/p>\r\n   <\/introduction>\r\n   <h3>Contents<\/h3>\r\n   <div>\r\n      <ul>\r\n         <li><a href=\"#1\">Get\/display image<\/a><\/li>\r\n         <li><a href=\"#3\">Segmentation masks<\/a><\/li>\r\n         <li><a href=\"#4\">Decide on a color plane\/colorspace<\/a><\/li>\r\n         <li><a href=\"#6\">The Three Amigos<\/a><\/li>\r\n         <li><a href=\"#7\">Now we can clean that up a bit<\/a><\/li>\r\n         <li><a href=\"#8\">Oops...lost Jiro's hair!<\/a><\/li>\r\n         <li><a href=\"#9\">Reconstruct Mask<\/a><\/li>\r\n         <li><a href=\"#10\">Get rid of the road and clean things up a bit<\/a><\/li>\r\n         <li><a href=\"#11\">Three Amigos in color<\/a><\/li>\r\n         <li><a href=\"#12\">Individual masks<\/a><\/li>\r\n         <li><a href=\"#13\">Three Amigos in pseudocolor<\/a><\/li>\r\n         <li><a href=\"#14\">Again, look at some options for subsequent segmentation spaces<\/a><\/li>\r\n         <li><a href=\"#15\">Finding Bob<\/a><\/li>\r\n         <li><a href=\"#16\">Now Bob's face...<\/a><\/li>\r\n         <li><a href=\"#17\">Reconstructing Bob<\/a><\/li>\r\n         <li><a href=\"#18\">Jiro and Brett...<\/a><\/li>\r\n         <li><a href=\"#19\">Jiro, and Brett<\/a><\/li>\r\n         <li><a href=\"#20\">Now let's shift the rgb images to close the \"Bob gap\"<\/a><\/li>\r\n         <li><a href=\"#21\">Now we can move Jiro and Brett into the space Bob once occupied<\/a><\/li>\r\n         <li><a href=\"#22\">Two Amigos, in color<\/a><\/li>\r\n         <li><a href=\"#23\">Add a background back in<\/a><\/li>\r\n         <li><a href=\"#24\">Finally, we drop Jiro and Brett into the new background<\/a><\/li>\r\n         <li><a href=\"#25\">One more bit of cleanup:<\/a><\/li>\r\n         <li><a href=\"#26\">Final result<\/a><\/li>\r\n      <\/ul>\r\n   <\/div>\r\n   <h3>Get\/display image<a name=\"1\"><\/a><\/h3><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">url = <span style=\"color: #A020F0\">'https:\/\/blogs.mathworks.com\/images\/pick\/threeamigos-800w.jpg'<\/span>;\r\noriginal = imread(url);\r\ntogglefig(<span style=\"color: #A020F0\">'Original'<\/span>)\r\nimshow(original);<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_01.jpg\"> <p>NOTE: <a href=\"https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/18220-create-and-activate-figures-by-name\">togglefig<\/a> is a helper function I use often. If a figure with the specified title exists, activate and use it. Otherwise, create it.\r\n   <\/p>\r\n   <h3>Segmentation masks<a name=\"3\"><\/a><\/h3>\r\n   <p>My approach is going to start with creating a segmentation mask of Jiro, Bob, and Brett, and then creating individual segmentation\r\n      masks for each.\r\n   <\/p>\r\n   <h3>Decide on a color plane\/colorspace<a name=\"4\"><\/a><\/h3>\r\n   <p>First, let's explore the information in the R,G, and B planes individually, as well as in the HSV, YCbCr, and LAB color spaces.<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">exploreRGB(original,<span style=\"color: #A020F0\">'advanced'<\/span>)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_02.jpg\"> <p>Here's a <a href=\"https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/19-delta-sigma-toolbox-explorergb\">link to exploreRGB<\/a>.\r\n   <\/p>\r\n   <h3>The Three Amigos<a name=\"6\"><\/a><\/h3>\r\n   <p>Let's segment the Three Amigos in the LAB colorspace<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">cform = makecform(<span style=\"color: #A020F0\">'srgb2lab'<\/span>);\r\n\r\n<span style=\"color: #228B22\">% A first pass at noise reduction using a median filter, operating plane-by-plane:<\/span>\r\n<span style=\"color: #0000FF\">for<\/span> ii = 1:3\r\n    original(:,:,ii) = medfilt2(original(:,:,ii),[6 6]);\r\n<span style=\"color: #0000FF\">end<\/span>\r\nlab = applycform(original,cform);\r\n\r\n<span style=\"color: #228B22\">% Now we're going to threshold the \"a*\" colorspace:<\/span>\r\na = lab(:,:,2);\r\nthreeAmigos = im2bw(a,0.5);\r\ntogglefig(<span style=\"color: #A020F0\">'Three Amigos'<\/span>)\r\nimshow(threeAmigos)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_03.jpg\"> <h3>Now we can clean that up a bit<a name=\"7\"><\/a><\/h3>\r\n   <p>Fill holes...<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">threeAmigos = imfill(threeAmigos,<span style=\"color: #A020F0\">'holes'<\/span>);\r\n<span style=\"color: #228B22\">% ...calculate connected components...<\/span>\r\ncc = bwconncomp(threeAmigos);\r\n<span style=\"color: #228B22\">% ...calculate the areas of all blobs in the image...<\/span>\r\nstats = regionprops(cc,<span style=\"color: #A020F0\">'Area'<\/span>);\r\n<span style=\"color: #228B22\">%...and keep only the largest:<\/span>\r\nA = [stats.Area];\r\n[~,biggest] = max(A);\r\nthreeAmigos(labelmatrix(cc)~=biggest) = 0;\r\ntogglefig(<span style=\"color: #A020F0\">'Three Amigos'<\/span>)\r\nimshow(threeAmigos)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_04.jpg\"> <h3>Oops...lost Jiro's hair!<a name=\"8\"><\/a><\/h3>\r\n   <p>That did a reasonable job of segmenting the Three Amigos. But I seem to have lost Jiro's hair, and he might want it back.\r\n      Let's go back and find it....\r\n   <\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">jirosHair = original(:,:,3) &lt;= 25.5;\r\n<span style=\"color: #228B22\">% Get rid of \"small\" (defined here as &lt;= 1050-pixel) blobs<\/span>\r\njirosHair = bwareaopen(jirosHair,1050);\r\ntogglefig(<span style=\"color: #A020F0\">'Jiro''s Hair'<\/span>)\r\nimshow(jirosHair)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_05.jpg\"> <h3>Reconstruct Mask<a name=\"9\"><\/a><\/h3>\r\n   <p>Now we can reconstruct the Three Amigos, in all their hairy (or hairless) glory<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\"><span style=\"color: #228B22\">% Binary OR:<\/span>\r\nthreeAmigos = threeAmigos | jirosHair;\r\ntogglefig(<span style=\"color: #A020F0\">'Three Amigos'<\/span>)\r\nimshow(threeAmigos)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_06.jpg\"> <h3>Get rid of the road and clean things up a bit<a name=\"10\"><\/a><\/h3>\r\n   <p>...by morphologically opening, then closing, the image:<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">threeAmigos = imopen(threeAmigos,strel(<span style=\"color: #A020F0\">'disk'<\/span>,19));\r\nthreeAmigos = imerode(threeAmigos,strel(<span style=\"color: #A020F0\">'disk'<\/span>,1));\r\ntogglefig(<span style=\"color: #A020F0\">'Three Amigos'<\/span>)\r\nimshow(threeAmigos)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_07.jpg\"> <h3>Three Amigos in color<a name=\"11\"><\/a><\/h3>\r\n   <p>Mask the background; here we set to zero all pixels outside of the threeAmigos mask using <a href=\"https:\/\/www.mathworks.com\/help\/releases\/R2010b\/techdoc\/math\/f1-85462.html#matrices_indexing_logicals\">logical indexing<\/a>:\r\n   <\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">amigosColor = original;\r\nr = amigosColor(:,:,1);\r\nr(~threeAmigos)= 0;\r\ng = amigosColor(:,:,2);\r\ng(~threeAmigos)= 0;\r\nb = amigosColor(:,:,3);\r\nb(~threeAmigos)= 0;\r\n<span style=\"color: #228B22\">% Concatenate in the \"3rd dimension\" to re-constitute an RGB image.<\/span>\r\namigosColor = cat(3,r,g,b);\r\ntogglefig(<span style=\"color: #A020F0\">'Amigos in Color'<\/span>)\r\nimshow(amigosColor)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_08.jpg\"> <h3>Individual masks<a name=\"12\"><\/a><\/h3>\r\n   <p>Now we can concentrate on segmenting individual masks<\/p>\r\n   <h3>Three Amigos in pseudocolor<a name=\"13\"><\/a><\/h3>\r\n   <p>Let's exaggerate the differences to make this easier<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">amigosColor = decorrstretch(amigosColor);\r\ntogglefig(<span style=\"color: #A020F0\">'Amigos in Pseudocolor'<\/span>)\r\nimshow(amigosColor)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_09.jpg\"> <h3>Again, look at some options for subsequent segmentation spaces<a name=\"14\"><\/a><\/h3><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">exploreRGB(amigosColor,<span style=\"color: #A020F0\">'advanced'<\/span>);<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_10.jpg\"> <h3>Finding Bob<a name=\"15\"><\/a><\/h3>\r\n   <p>Let's again work in the LAB colorspace<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">lab2 = applycform(amigosColor,cform);\r\n<span style=\"color: #228B22\">% This time, we will use the \"b* colorspace\"<\/span>\r\nb2 = lab2(:,:,3);\r\n<span style=\"color: #228B22\">% Segment Bob:<\/span>\r\nbobBodyHair = ~im2bw(b2,0.47);\r\nbobBodyHair = imclose(bobBodyHair,strel(<span style=\"color: #A020F0\">'disk'<\/span>,3));\r\nbobBodyHair = bwareaopen(bobBodyHair,50);\r\n\r\n<span style=\"color: #228B22\">%Bob is in the middle; let's keep only the two \"middle\" regions (let's say,<\/span>\r\n<span style=\"color: #228B22\">%within 30 pixels of the center)<\/span>\r\ncc = bwconncomp(bobBodyHair);\r\nstats = regionprops(cc,<span style=\"color: #A020F0\">'Centroid'<\/span>);\r\nC = reshape([stats.Centroid],2,[])';\r\ninds = find(C(:,1) &gt; size(bobBodyHair,2)\/2 - 30 &amp;  C(:,1) &lt; size(bobBodyHair,2)\/2 + 30);\r\nbobBodyHair = ismember(labelmatrix(cc), inds);\r\ntogglefig(<span style=\"color: #A020F0\">'Finding Bob'<\/span>)\r\nimshow(bobBodyHair)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_11.jpg\"> <h3>Now Bob's face...<a name=\"16\"><\/a><\/h3>\r\n   <p>Still working with \"b*,\" but using a different threshold:<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">bobFace = im2bw(b2,0.56);\r\nbobFace = imclearborder(bobFace);\r\ncc = bwconncomp(bobFace);\r\nstats = regionprops(cc,<span style=\"color: #A020F0\">'Area'<\/span>);\r\nA = [stats.Area];\r\n[~,biggest] = max(A);\r\nbobFace(labelmatrix(cc)~=biggest) = 0;\r\ntogglefig(<span style=\"color: #A020F0\">'Finding Bob'<\/span>)\r\nimshow(bobFace)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_12.jpg\"> <h3>Reconstructing Bob<a name=\"17\"><\/a><\/h3><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">bob = bobBodyHair | bobFace;\r\nbob = imfill(bob,<span style=\"color: #A020F0\">'holes'<\/span>);\r\ntogglefig(<span style=\"color: #A020F0\">'Finding Bob'<\/span>)\r\nimshow(bob)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_13.jpg\"> <h3>Jiro and Brett...<a name=\"18\"><\/a><\/h3><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">jiroBrett = threeAmigos &amp; ~bob;\r\njiroBrett = bwareaopen(jiroBrett,300);\r\ntogglefig(<span style=\"color: #A020F0\">'Jiro and Brett'<\/span>)\r\nimshow(jiroBrett)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_14.jpg\"> <h3>Jiro, and Brett<a name=\"19\"><\/a><\/h3>\r\n   <p>The connected components matrix is very convenient here:<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">cc = bwconncomp(jiroBrett);\r\njiro = labelmatrix(cc) == 1;\r\nbrett = labelmatrix(cc) == 2;\r\n\r\ntogglefig(<span style=\"color: #A020F0\">'Jiro'<\/span>)\r\nimshow(jiro);\r\ntitle(<span style=\"color: #A020F0\">'Jiro'<\/span>)\r\ntogglefig(<span style=\"color: #A020F0\">'Brett'<\/span>)\r\nimshow(brett)\r\ntitle(<span style=\"color: #A020F0\">'Brett'<\/span>)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_15.jpg\"> <img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_16.jpg\"> <h3>Now let's shift the rgb images to close the \"Bob gap\"<a name=\"20\"><\/a><\/h3>\r\n   <p>How \"wide\" is Bob at the bottom of the image? (Plus a bit of padding.)<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">shiftAmt = floor(nnz(bob(end,:))\/2) + 7;<\/pre><h3>Now we can move Jiro and Brett into the space Bob once occupied<a name=\"21\"><\/a><\/h3><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">twoAmigos = circshift(jiro,[0 shiftAmt]) | circshift(brett,[0,-shiftAmt]);\r\ntogglefig(<span style=\"color: #A020F0\">'Two Amigos'<\/span>);\r\nimshow(twoAmigos);<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_17.jpg\"> <h3>Two Amigos, in color<a name=\"22\"><\/a><\/h3>\r\n   <p>We can now create a color image that excludes Bob<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\"><span style=\"color: #228B22\">% (We re-read the original because we reduced noise in it above using median<\/span>\r\n<span style=\"color: #228B22\">% filtering, prior to masking. Now we want the unsmoothed original back.)<\/span>\r\noriginal = imread(url);\r\n\r\nrJiro = original(:,:,1);\r\nrJiro(~jiro) = 0;\r\nrJiro = circshift(rJiro,[0 shiftAmt]);\r\ngJiro = original(:,:,2);\r\ngJiro(~jiro) = 0;\r\ngJiro = circshift(gJiro,[0 shiftAmt]);\r\nbJiro = original(:,:,3);\r\nbJiro(~jiro) = 0;\r\nbJiro = circshift(bJiro,[0 shiftAmt]);\r\n\r\nrBrett = original(:,:,1);\r\nrBrett(~brett) = 0;\r\nrBrett = circshift(rBrett,[0 -shiftAmt]);\r\ngBrett = original(:,:,2);\r\ngBrett(~brett) = 0;\r\ngBrett = circshift(gBrett,[0 -shiftAmt]);\r\nbBrett = original(:,:,3);\r\nbBrett(~brett) = 0;\r\nbBrett = circshift(bBrett,[0 -shiftAmt]);\r\n\r\nr = max(rJiro,rBrett);\r\ng = max(gJiro,gBrett);\r\nb = max(bJiro,bBrett);\r\nrgb = cat(3,r,g,b);\r\ntogglefig(<span style=\"color: #A020F0\">'Two Amigos Color'<\/span>)\r\nimshow(rgb)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_18.jpg\"> <h3>Add a background back in<a name=\"23\"><\/a><\/h3>\r\n   <p>Rather than try to \"fix\" the original background, I'm going to select a new backdrop for the \"Two Amigos\": ('forest.tif' is\r\n      a sample image in the <a href=\"https:\/\/www.mathworks.com\/products\/image\/\">Image Processing Toolbox<\/a>.)\r\n   <\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">[bg,map] = imread(<span style=\"color: #A020F0\">'forest.tif'<\/span>);\r\nbg = ind2rgb(bg,map);\r\n<span style=\"color: #228B22\">% Resize it...<\/span>\r\nbg = imresize(bg,size(r,1)\/size(bg,1),<span style=\"color: #A020F0\">'bicubic'<\/span>);\r\nbg = bg(:,1:size(r,2),:);\r\nbg = im2uint8(bg);\r\ntogglefig(<span style=\"color: #A020F0\">'(New) Background'<\/span>)\r\nimshow(bg)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_19.jpg\"> <h3>Finally, we drop Jiro and Brett into the new background<a name=\"24\"><\/a><\/h3><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">bgR = bg(:,:,1);\r\nbgR(twoAmigos) = 0;\r\nbgG = bg(:,:,2);\r\nbgG(twoAmigos) = 0;\r\nbgB = bg(:,:,3);\r\nbgB(twoAmigos) = 0;\r\nr = max(r,bgR);\r\ng = max(g,bgG);\r\nb = max(b,bgB);\r\nrgb = cat(3,r,g,b);\r\ntogglefig(<span style=\"color: #A020F0\">'Two Amigos Color'<\/span>)\r\nimshow(rgb)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_20.jpg\"> <h3>One more bit of cleanup:<a name=\"25\"><\/a><\/h3>\r\n   <p>The edge is a bit rough...<\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">brettJiroEdge = bwperim(twoAmigos);\r\nbrettJiroEdge = imdilate(brettJiroEdge,strel(<span style=\"color: #A020F0\">'disk'<\/span>,9));\r\ntogglefig(<span style=\"color: #A020F0\">'Two Amigos Outline'<\/span>)\r\nimshow(brettJiroEdge)<\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_21.jpg\"> <h3>Final result<a name=\"26\"><\/a><\/h3>\r\n   <p>Soften the edges: we can soften those edges a bit by applying locally an averaging filter where the (dilated) outline is logically\r\n      true:\r\n   <\/p><pre style=\"background: #F9F7F3; padding: 10px; border: 1px solid rgb(200,200,200)\">H = fspecial(<span style=\"color: #A020F0\">'average'<\/span>);\r\n<span style=\"color: #228B22\">% Roifilt2 is a 2-D filter. We can use it on RGB images, but we have to<\/span>\r\n<span style=\"color: #228B22\">% operate plane-by-plane. But first, it's good practice to preallocate the<\/span>\r\n<span style=\"color: #228B22\">% final matrix:<\/span>\r\nFINAL = zeros(size(rgb),<span style=\"color: #A020F0\">'uint8'<\/span>);\r\n<span style=\"color: #0000FF\">for<\/span> ii = 1:3\r\n    FINAL(:,:,ii) = roifilt2(H,rgb(:,:,ii),brettJiroEdge);\r\n<span style=\"color: #0000FF\">end<\/span>\r\nFINAL = imcrop(FINAL,[58 90 691 471]);\r\ntogglefig(<span style=\"color: #A020F0\">'Final Result'<\/span>)\r\nimshow(FINAL)\r\n\r\n\r\n<span style=\"color: #228B22\">% et voila!<\/span><\/pre><img decoding=\"async\" vspace=\"5\" hspace=\"5\" src=\"https:\/\/blogs.mathworks.com\/images\/steve\/2010\/TwoAmigos_BDS_22.jpg\"> <script language=\"JavaScript\">\r\n<!--\r\n\r\n    function grabCode_7d17e51312c14dd8a3dc5807a615cce8() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='7d17e51312c14dd8a3dc5807a615cce8 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' 7d17e51312c14dd8a3dc5807a615cce8';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        author = 'Steve Eddins';\r\n        copyright = 'Copyright 2010 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add author and copyright lines at the bottom if specified.\r\n        if ((author.length > 0) || (copyright.length > 0)) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (author.length > 0) {\r\n                d.writeln('% _' + author + '_');\r\n            }\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n      \r\n      d.title = title + ' (MATLAB code)';\r\n      d.close();\r\n      }   \r\n      \r\n-->\r\n<\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\"><br><a href=\"javascript:grabCode_7d17e51312c14dd8a3dc5807a615cce8()\"><span style=\"font-size: x-small;        font-style: italic;\">Get \r\n            the MATLAB code \r\n            <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      Published with MATLAB&reg; 7.11<br><\/p>\r\n<\/div>\r\n<!--\r\n7d17e51312c14dd8a3dc5807a615cce8 ##### SOURCE BEGIN #####\r\n%% Image Magic: The Two Amigos \r\n% I'd like to introduce Brett Shoelson as guest blogger for today. Many of\r\n% you already know Brett from his work on the \r\n% <https:\/\/blogs.mathworks.com\/pick\/ File Exchange Pick of the Week blog>,\r\n% or his posts on the \r\n% <http:\/\/ comp.soft-sys.matlab>\r\n% newsgroup, or from his many presentations given on the road. I met Brett\r\n% \"virtually\" many years ago, long before he came to work for MathWorks,\r\n% through his posts on the newsgroup. Brett's been doing fun, interesting,\r\n% and even useful things to images using MATLAB for quite a long time.\r\n%\r\n% A few weeks ago Brett posted the\r\n% <https:\/\/blogs.mathworks.com\/pick\/2010\/09\/03\/two-amigos\/ \r\n% Two Amigos Challenge> on the Pick of the Week blog. Readers of the blog\r\n% know that long-time contributor Bob Bemis (Hi, Bob!) has moved on from\r\n% MathWorks and isn't posting there anymore. Brett challenged his readers\r\n% to digitally retouch the \"Three Amigos\" picture on the blog and turn it\r\n% into a \"Two Amigos\" picture using only MathWorks products.\r\n%\r\n% I was so impressed with Brett's own solution to the Challenge that I\r\n% invited him to show it to my readers here.\r\n%\r\n% So here is Brett's solution to the Two Amigos Challenge ...\r\n%\r\n% ... in which we remove a subject from a photograph, and digitally retouch it\r\n% to conceal the modification...\r\n\r\n%% Get\/display image\r\nurl = 'https:\/\/blogs.mathworks.com\/images\/pick\/threeamigos-800w.jpg';\r\noriginal = imread(url);\r\ntogglefig('Original')\r\nimshow(original);\r\n\r\n%%\r\n% NOTE:\r\n% <https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/18220-create-and-activate-figures-by-name togglefig>\r\n% is a helper function I use often. If a figure with the specified title\r\n% exists, activate and use it. Otherwise, create it.  \r\n\r\n%% Segmentation masks\r\n% My approach is going to start with creating a segmentation mask of Jiro,\r\n% Bob, and Brett, and then creating individual segmentation masks for each. \r\n\r\n%% Decide on a color plane\/colorspace\r\n% First, let's explore the information in the R,G, and B planes\r\n% individually, as well as in the HSV, YCbCr, and LAB color spaces. \r\nexploreRGB(original,'advanced')\r\n\r\n%%\r\n% Here's a <https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/19-delta-sigma-toolbox-explorergb link to exploreRGB>.\r\n\r\n%% The Three Amigos\r\n% Let's segment the Three Amigos in the LAB colorspace\r\ncform = makecform('srgb2lab');\r\n\r\n% A first pass at noise reduction using a median filter, operating plane-by-plane:\r\nfor ii = 1:3\r\n    original(:,:,ii) = medfilt2(original(:,:,ii),[6 6]);\r\nend\r\nlab = applycform(original,cform);\r\n\r\n% Now we're going to threshold the \"a*\" colorspace:\r\na = lab(:,:,2);\r\nthreeAmigos = im2bw(a,0.5);\r\ntogglefig('Three Amigos')\r\nimshow(threeAmigos)\r\n\r\n%% Now we can clean that up a bit\r\n% Fill holes...\r\nthreeAmigos = imfill(threeAmigos,'holes');\r\n% ...calculate connected components...\r\ncc = bwconncomp(threeAmigos);\r\n% ...calculate the areas of all blobs in the image... \r\nstats = regionprops(cc,'Area');\r\n%...and keep only the largest:\r\nA = [stats.Area];\r\n[~,biggest] = max(A);\r\nthreeAmigos(labelmatrix(cc)~=biggest) = 0;\r\ntogglefig('Three Amigos')\r\nimshow(threeAmigos)\r\n\r\n%% Oops...lost Jiro's hair!\r\n% That did a reasonable job of segmenting the Three Amigos. But I seem to\r\n% have lost Jiro's hair, and he might want it back. Let's go back and find\r\n% it....  \r\n\r\njirosHair = original(:,:,3) <= 25.5;\r\n% Get rid of \"small\" (defined here as <= 1050-pixel) blobs\r\njirosHair = bwareaopen(jirosHair,1050);\r\ntogglefig('Jiro''s Hair')\r\nimshow(jirosHair)\r\n\r\n%% Reconstruct Mask\r\n% Now we can reconstruct the Three Amigos, in all their hairy (or hairless)\r\n% glory\r\n\r\n% Binary OR:\r\nthreeAmigos = threeAmigos | jirosHair;\r\ntogglefig('Three Amigos')\r\nimshow(threeAmigos)\r\n\r\n%% Get rid of the road and clean things up a bit\r\n% ...by morphologically opening, then closing, the image:\r\nthreeAmigos = imopen(threeAmigos,strel('disk',19));\r\nthreeAmigos = imerode(threeAmigos,strel('disk',1));\r\ntogglefig('Three Amigos')\r\nimshow(threeAmigos)\r\n\r\n%% Three Amigos in color\r\n% Mask the background; here we set to zero all pixels outside of the\r\n% threeAmigos mask using \r\n% <https:\/\/www.mathworks.com\/help\/releases\/R2010b\/techdoc\/math\/f1-85462.html#matrices_indexing_logicals logical indexing>:\r\namigosColor = original;\r\nr = amigosColor(:,:,1);\r\nr(~threeAmigos)= 0;\r\ng = amigosColor(:,:,2);\r\ng(~threeAmigos)= 0;\r\nb = amigosColor(:,:,3);\r\nb(~threeAmigos)= 0;\r\n% Concatenate in the \"3rd dimension\" to re-constitute an RGB image.\r\namigosColor = cat(3,r,g,b);\r\ntogglefig('Amigos in Color')\r\nimshow(amigosColor)\r\n\r\n%% Individual masks\r\n% Now we can concentrate on segmenting individual masks\r\n\r\n%% Three Amigos in pseudocolor\r\n% Let's exaggerate the differences to make this easier\r\namigosColor = decorrstretch(amigosColor);\r\ntogglefig('Amigos in Pseudocolor')\r\nimshow(amigosColor)\r\n\r\n%% Again, look at some options for subsequent segmentation spaces\r\nexploreRGB(amigosColor,'advanced');\r\n\r\n%% Finding Bob\r\n% Let's again work in the LAB colorspace\r\nlab2 = applycform(amigosColor,cform);\r\n% This time, we will use the \"b* colorspace\"\r\nb2 = lab2(:,:,3);\r\n% Segment Bob:\r\nbobBodyHair = ~im2bw(b2,0.47);\r\nbobBodyHair = imclose(bobBodyHair,strel('disk',3));\r\nbobBodyHair = bwareaopen(bobBodyHair,50);\r\n\r\n%Bob is in the middle; let's keep only the two \"middle\" regions (let's say,\r\n%within 30 pixels of the center)\r\ncc = bwconncomp(bobBodyHair);\r\nstats = regionprops(cc,'Centroid');\r\nC = reshape([stats.Centroid],2,[])';\r\ninds = find(C(:,1) > size(bobBodyHair,2)\/2 - 30 &  C(:,1) < size(bobBodyHair,2)\/2 + 30);\r\nbobBodyHair = ismember(labelmatrix(cc), inds);\r\ntogglefig('Finding Bob')\r\nimshow(bobBodyHair)\r\n\r\n%% Now Bob's face...\r\n% Still working with \"b*,\" but using a different threshold:\r\nbobFace = im2bw(b2,0.56);\r\nbobFace = imclearborder(bobFace);\r\ncc = bwconncomp(bobFace);\r\nstats = regionprops(cc,'Area');\r\nA = [stats.Area];\r\n[~,biggest] = max(A);\r\nbobFace(labelmatrix(cc)~=biggest) = 0;\r\ntogglefig('Finding Bob')\r\nimshow(bobFace)\r\n\r\n%% Reconstructing Bob\r\nbob = bobBodyHair | bobFace;\r\nbob = imfill(bob,'holes');\r\ntogglefig('Finding Bob')\r\nimshow(bob)\r\n\r\n%% Jiro and Brett...\r\njiroBrett = threeAmigos & ~bob;\r\njiroBrett = bwareaopen(jiroBrett,300);\r\ntogglefig('Jiro and Brett')\r\nimshow(jiroBrett)\r\n\r\n%% Jiro, and Brett\r\n% The connected components matrix is very convenient here:\r\ncc = bwconncomp(jiroBrett);\r\njiro = labelmatrix(cc) == 1;\r\nbrett = labelmatrix(cc) == 2;\r\n\r\ntogglefig('Jiro')\r\nimshow(jiro);\r\ntitle('Jiro')\r\ntogglefig('Brett')\r\nimshow(brett)\r\ntitle('Brett')\r\n\r\n%% Now let's shift the rgb images to close the \"Bob gap\"\r\n% How \"wide\" is Bob at the bottom of the image? (Plus a bit of padding.)\r\nshiftAmt = floor(nnz(bob(end,:))\/2) + 7;\r\n\r\n%% Now we can move Jiro and Brett into the space Bob once occupied\r\ntwoAmigos = circshift(jiro,[0 shiftAmt]) | circshift(brett,[0,-shiftAmt]);\r\ntogglefig('Two Amigos');\r\nimshow(twoAmigos);\r\n\r\n%% Two Amigos, in color\r\n% We can now create a color image that excludes Bob\r\n\r\n% (We re-read the original because we reduced noise in it above using median\r\n% filtering, prior to masking. Now we want the unsmoothed original back.)\r\noriginal = imread(url);\r\n\r\nrJiro = original(:,:,1);\r\nrJiro(~jiro) = 0;\r\nrJiro = circshift(rJiro,[0 shiftAmt]);\r\ngJiro = original(:,:,2);\r\ngJiro(~jiro) = 0;\r\ngJiro = circshift(gJiro,[0 shiftAmt]);\r\nbJiro = original(:,:,3);\r\nbJiro(~jiro) = 0;\r\nbJiro = circshift(bJiro,[0 shiftAmt]);\r\n\r\nrBrett = original(:,:,1);\r\nrBrett(~brett) = 0;\r\nrBrett = circshift(rBrett,[0 -shiftAmt]);\r\ngBrett = original(:,:,2);\r\ngBrett(~brett) = 0;\r\ngBrett = circshift(gBrett,[0 -shiftAmt]);\r\nbBrett = original(:,:,3);\r\nbBrett(~brett) = 0;\r\nbBrett = circshift(bBrett,[0 -shiftAmt]);\r\n\r\nr = max(rJiro,rBrett);\r\ng = max(gJiro,gBrett);\r\nb = max(bJiro,bBrett);\r\nrgb = cat(3,r,g,b);\r\ntogglefig('Two Amigos Color')\r\nimshow(rgb)\r\n\r\n%% Add a background back in\r\n% Rather than try to \"fix\" the original background, I'm going to select a\r\n% new backdrop for the \"Two Amigos\":\r\n% ('forest.tif' is a sample image in the <https:\/\/www.mathworks.com\/products\/image\/ Image Processing Toolbox>.)\r\n[bg,map] = imread('forest.tif');\r\nbg = ind2rgb(bg,map);\r\n% Resize it...\r\nbg = imresize(bg,size(r,1)\/size(bg,1),'bicubic');\r\nbg = bg(:,1:size(r,2),:);\r\nbg = im2uint8(bg);\r\ntogglefig('(New) Background')\r\nimshow(bg)\r\n\r\n%% Finally, we drop Jiro and Brett into the new background\r\nbgR = bg(:,:,1);\r\nbgR(twoAmigos) = 0;\r\nbgG = bg(:,:,2);\r\nbgG(twoAmigos) = 0;\r\nbgB = bg(:,:,3);\r\nbgB(twoAmigos) = 0;\r\nr = max(r,bgR);\r\ng = max(g,bgG);\r\nb = max(b,bgB);\r\nrgb = cat(3,r,g,b);\r\ntogglefig('Two Amigos Color')\r\nimshow(rgb)\r\n\r\n%% One more bit of cleanup:\r\n% The edge is a bit rough...\r\nbrettJiroEdge = bwperim(twoAmigos);\r\nbrettJiroEdge = imdilate(brettJiroEdge,strel('disk',9));\r\ntogglefig('Two Amigos Outline')\r\nimshow(brettJiroEdge)\r\n\r\n%% Final result\r\n% Soften the edges: we can soften those edges a bit by applying locally an\r\n% averaging filter where the (dilated) outline is logically true:  \r\nH = fspecial('average');\r\n% Roifilt2 is a 2-D filter. We can use it on RGB images, but we have to\r\n% operate plane-by-plane. But first, it's good practice to preallocate the\r\n% final matrix: \r\nFINAL = zeros(size(rgb),'uint8');\r\nfor ii = 1:3\r\n    FINAL(:,:,ii) = roifilt2(H,rgb(:,:,ii),brettJiroEdge);\r\nend\r\nFINAL = imcrop(FINAL,[58 90 691 471]);\r\ntogglefig('Final Result')\r\nimshow(FINAL)\r\n\r\n\r\n% et voila!\r\n##### SOURCE END ##### 7d17e51312c14dd8a3dc5807a615cce8\r\n-->","protected":false},"excerpt":{"rendered":"<p>\r\n   \r\n      I'd like to introduce Brett Shoelson as guest blogger for today. Many of you already know Brett from his work on the File Exchange Pick of the Week blog, or his posts on the... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/steve\/2010\/10\/08\/the-two-amigos\/\">read more >><\/a><\/p>","protected":false},"author":42,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[114,138,561,140,46,434,537,348,252,292,84,452,404,144,124,246,136,108,76,156,36,384,422,563,112,122,699,707,168,170,709,190,106,52,130],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts\/344"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/users\/42"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/comments?post=344"}],"version-history":[{"count":1,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts\/344\/revisions"}],"predecessor-version":[{"id":2754,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/posts\/344\/revisions\/2754"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/media?parent=344"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/categories?post=344"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/steve\/wp-json\/wp\/v2\/tags?post=344"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}