{"id":66,"date":"2017-12-15T07:00:21","date_gmt":"2017-12-15T07:00:21","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=66"},"modified":"2021-04-06T15:52:36","modified_gmt":"2021-04-06T19:52:36","slug":"network-visualization-based-on-occlusion-sensitivity","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2017\/12\/15\/network-visualization-based-on-occlusion-sensitivity\/","title":{"rendered":"Network Visualization Based on Occlusion Sensitivity"},"content":{"rendered":"

Have you ever wondered what your favorite deep learning network is looking at? For example, if a network classifies this image as \"French horn,\" what part of the image matters most for the classification?<\/p>

\"\" <\/p>

Birju Patel, a developer on the Computer Vision System Toolbox team, helped me with the main idea and code for today's post. Birju has focused on deep learning for the last couple of years. Before that, he worked on feature extraction methods and on optimizing feature matching.<\/p>

Let's use the pretrained ResNet-50 network for this experiment. (He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, Sun, Jian. \"Deep Residual Learning for Image Recognition.\" In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778. 2016.) An easy way to get the ResNet-50 network for MATLAB is to launch the Add-On Explorer (from the HOME tab in MATLAB) and search for resnet.<\/p>

net = resnet50;\r\n<\/pre>
We need to be aware that ResNet-50 expects the input images to be a particular size. The network's initial layer has this information.<\/p>
sz = net.Layers(1).InputSize(1:2)\r\n<\/pre>
\r\nsz =\r\n\r\n   224   224\r\n\r\n<\/pre>
The required image size can be passed directly to the imresize<\/tt> function.<\/p>
url = 'https:\/\/blogs.mathworks.com\/steve\/files\/steve-horn.jpg'<\/span>;\r\nrgb = imread(url);\r\nrgb = imresize(rgb,sz);\r\nimshow(rgb)\r\n<\/pre>\"\" 
Call classify<\/tt> with the network and the image to see what category the network thinks is most probable.<\/p>
classify(net,rgb)\r\n<\/pre>
\r\nans = \r\n\r\n  categorical\r\n\r\n     French horn \r\n\r\n<\/pre>
ResNet-50 thinks I am playing the French horn.<\/p>
Birju was reading a paper by Zeiler and Fergus<\/a> about visualization techniques for convolutional neural networks, and in it he came across the idea of occlusion sensitivity<\/i>. If you block out, or occlude, a portion of the image, how does that affect the probability score of the network? And how does the result vary depending on which portion you occlude?<\/p>
Let's try it.<\/p>
rgb2 = rgb;\r\nrgb2((1:71)+77,(1:71)+108,:) = 128;\r\nimshow(rgb2)\r\n<\/pre>\"\" 
classify(net,rgb2)\r\n<\/pre>
\r\nans = \r\n\r\n  categorical\r\n\r\n     notebook \r\n\r\n<\/pre>
Hmm. I guess the network \"thinks\" that gray square looks like a notebook. That region must be important for classifying the image. Now let's try the occlusion in a different spot.<\/p>
rgb3 = rgb;\r\nrgb3((1:71)+15,(1:71)+80,:) = 128;\r\nimshow(rgb3)\r\n<\/pre>\"\" 
classify(net,rgb3)\r\n<\/pre>
\r\nans = \r\n\r\n  categorical\r\n\r\n     French horn \r\n\r\n<\/pre>
Hmm. I guess my head is not as important.<\/p>
Anyway, Birju wrote some MATLAB code to systematically quantify the relative importance of different images regions to the classification result. His code builds up a large batch of images. For each image in the batch, a different region is occluded. For each location of the occlusion mask, the prediction score of the expected class (\"French horn,\" in this case) is recorded.<\/p>
Let's make a batch of images with 71x71 regions masked out. Start by computing the corners of all the masks, represented as (X1,Y1) and (X2,Y2).<\/p>
mask_size = [71 71];\r\n[H,W,~] = size(rgb);\r\n\r\nX = 1:W;\r\nY = 1:H;\r\n\r\n[X1, Y1] = meshgrid(X, Y);\r\n\r\nX1 = X1(:) - (mask_size(2)-1)\/2;\r\nY1 = Y1(:) - (mask_size(1)-1)\/2;\r\n\r\nX2 = X1 + mask_size(2) - 1;\r\nY2 = Y1 + mask_size(1) - 1;\r\n<\/pre>
Don't let the mask corners stray outside the image boundaries.<\/p>
X1 = max(1, X1);\r\nY1 = max(1, Y1);\r\n\r\nX2 = min(W, X2);\r\nY2 = min(H, Y2);\r\n<\/pre>
Make the batch.<\/p>
batch = repmat(rgb,[1 1 1 size(X1,1)]);\r\n\r\nfor<\/span> i = 1:size(X1,1)\r\n   c = X1(i):X2(i);\r\n   r = Y1(i):Y2(i);\r\n   batch(r,c,:,i) = 128; % gray mask.<\/span>\r\nend<\/span>\r\n<\/pre>
[Note: This batch has more than 50,000 images in it. You'll need a lot of RAM to create and process such a large batch of images all at once.]<\/p>
Here are a few of the masked images.<\/p>
montage(batch(:,:,:,randperm(size(X1,1),9)))\r\n<\/pre>\"\" 
Now I'll use predict<\/tt> (instead of classify<\/tt>) to get the prediction scores for each category and for each image in the batch. The 'MiniBatchSize'<\/tt> parameter is used to keep the GPU memory use down. It means that the predict<\/tt> function will send 64 images at a time to the GPU for processing.<\/p>
s = predict(net, batch, 'MiniBatchSize'<\/span>,64);\r\n<\/pre>
size(s)\r\n<\/pre>
\r\nans =\r\n\r\n       50176        1000\r\n\r\n<\/pre>
That's a lot of prediction scores! There are 51,529 images in the batch, and there are 1,000 categories. The matrix s<\/tt> has a score for each category and for each image.<\/p>
We are specifically interested in the prediction scores for the category predicted for the original image. Let's figure out the category index for that.<\/p>
scores = predict(net,rgb);\r\n[~,horn_idx] = max(scores);\r\n<\/pre>
So, here are the French horn<\/i> scores for every image in the batch:<\/p>
s_horn = s(:,horn_idx);\r\n<\/pre>
Reshape the set of horn scores to be an image and display it.<\/p>
S_horn = reshape(s_horn,H,W);\r\nimshow(-S_horn,[])\r\ncolormap(gca,'parula'<\/span>)\r\n<\/pre>\"\" 
The brightest regions indicate the locations where the occlusion had the biggest effect on the probability score.<\/p>
Let's find the location that minimizes the \"French horn\" probability score.<\/p>
[min_score,min_idx] = min(s_horn);\r\nrgb_min_score = batch(:,:,:,min_idx);\r\nimshow(rgb_min_score)\r\n<\/pre>\"\" 
There you go. To recognize a French horn, it's all about the valves and valve slides. It's not about the bell.<\/p>
A final note on terminology: Some of my horn-playing friends might give me a hard time about calling my instrument a \"French horn.\" According to the International Horn Society, the instrument should just be called \"horn.\"<\/a> However, the label stored in ResNet-50 is \"French horn,\" and that is the most commonly used term in the United States, where I live.<\/p>