{"id":5835,"date":"2021-01-26T09:54:58","date_gmt":"2021-01-26T14:54:58","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=5835"},"modified":"2021-04-06T15:45:30","modified_gmt":"2021-04-06T19:45:30","slug":"deep-learning-visualizations","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2021\/01\/26\/deep-learning-visualizations\/","title":{"rendered":"Deep Learning Visualizations"},"content":{"rendered":"Evaluating deep learning model performance can be done a variety of ways. A confusion matrix answers some questions about the model performance, but not all. How do we know that the model is identifying the right features? <\/span>\r\n
<\/h6>\r\nLet's walk through some of the easy ways to explore deep learning models using visualization, with links to documentation examples for more information.<\/span>\r\n
<\/h6>\r\n

Background: Data and Model Information<\/h2>\r\n

<\/h6>\r\nFor these visualizations, I'm using models created by my colleague Heather Gorr. You can find the code she used to train the models on GitHub here<\/a><\/strong>.<\/span>\r\n\r\nThe basic premise of the models and code is using wildlife data to see if MATLAB can be used to correctly identify classes of animals in the wild. The sample images look something like this:<\/span>\r\n
<\/h6>\r\n\r\n\r\n\r\n\r\n
Bear\r\n\"\"<\/td>\r\nBear\r\n\r\n\"\"<\/td>\r\nBear\r\n\r\n\"\"<\/td>\r\nBighorn Sheep\r\n\r\n\"\"<\/td>\r\nBighorn Sheep\r\n\"\"<\/td>\r\n<\/tr>\r\n
Cow\r\n\"\"<\/td>\r\nCow\r\n\r\n\"\"<\/td>\r\nCoyote\r\n\r\n\"\"<\/td>\r\nCoyote\r\n\r\n\"\"<\/td>\r\nDog\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\nThe two models I will use today were trained on the following classes of images:<\/span>\r\n
    \r\n \t
  1. \u00a0[ Bear | Not-Bear ] All bears should be classified as \"bear\", anything else should be \"not bear\"<\/span><\/li>\r\n \t
  2. \u00a0[ Bear | Cow | Sheep ]: 3 classes of animals that must fit into the categories<\/span><\/li>\r\n<\/ol>\r\nHere are the sample images run through each network.<\/span>\r\n
    <\/h6>\r\n

    Bear | Not-Bear Classifier (net1)<\/h3>\r\n\r\n\r\n\r\n\r\n
    bear\r\n\"\"<\/td>\r\nbear\r\n\r\n\"\"<\/td>\r\nbear\r\n\r\n\"\"<\/td>\r\nnot bear\r\n\r\n\"\"<\/td>\r\nnot bear\r\n\"\"<\/td>\r\n<\/tr>\r\n
    not bear\r\n\"\"<\/td>\r\nnot bear\r\n\r\n\"\"<\/td>\r\nnot bear\r\n\r\n\"\"<\/td>\r\nnot bear\r\n\r\n\"\"<\/td>\r\nnot bear\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

    Bear | Cow | Sheep Classifier (net2)<\/h3>\r\n\r\n\r\n\r\n\r\n
    bear\r\n\"\"<\/td>\r\nbear\r\n\r\n\"\"<\/td>\r\nbear\r\n\r\n\"\"<\/td>\r\nbighorn sheep\r\n\r\n\"\"<\/td>\r\nbighorn sheep\r\n\"\"<\/td>\r\n<\/tr>\r\n
    cow\r\n\"\"<\/td>\r\ncow\r\n\r\n\"\"<\/td>\r\nbear\r\n\r\n\"\"<\/td>\r\nbear\r\n\r\n\"\"<\/td>\r\ncow\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\nYou now can see that most predictions are correct, but once I add in a few animals not in the expected categories, the predictions get a little off. The rest of this post is using visualization to find why <\/em>the models predicted a certain way.<\/span>\r\n
    <\/h6>\r\n

    Popular Visualization Techniques<\/h1>\r\n

    ...and how to use them<\/h2>\r\nTechniques like LIME, Grad-CAM and Occlusion Sensitivity can give you insight into the network, and why the network chose a particular option.<\/span>\r\n
    <\/h6>\r\n

    LIME<\/h2>\r\n

    <\/h6>\r\nimageLIME<\/span> is newer technique added to MATLAB in 2020b. Let's investigate the coyote that was called a bear:<\/span>\r\n\r\n \r\n\r\n\r\n\r\n
    Start by displaying the predictions and the confidences.<\/span>\r\n
    <\/h6>\r\n
    [YPred,scores] = classify(net2,img);\r\n[~,topIdx] = maxk(scores, 3);\r\ntopScores = scores(topIdx);\r\ntopClasses = classes(topIdx);\r\n\r\nfigure; imshow(img)\r\ntitleString = compose(\"%s (%.2f)\",topClasses,topScores'); %'\r\ntitle(sprintf(join(titleString, \"; \")));<\/pre>\r\n<\/td>\r\n
    		\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\n\r\n\r\n\r\n\r\n
    Then, use imageLime<\/span> to visualize the output.<\/span>\r\n
    <\/h6>\r\n
    map = imageLIME(net2,img,YPred);\r\n\r\nfigure\r\nimshow(img,'InitialMagnification',150)\r\nhold on\r\nimagesc(map,'AlphaData',0.5)\r\ncolormap jet\r\ncolorbar\r\n\r\ntitle(sprintf(\"Image LIME (%s)\", YPred))\r\nhold off<\/pre>\r\n<\/td>\r\n
    		\"\"<\/td>\r\n<\/tr>\r\n
    \r\nHere, imageLIME is indicating the reason for the prediction is in the lower corner of the image. While this is clearly an incorrect prediction, the strongest features are not surrounding the coyote. This would indicate those features were learned incorrectly for the bear class.<\/span><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n\r\n
    <\/h6>\r\n

    Grad-CAM<\/h2>\r\nLet's focus on the bears using Grad-CAM, another visualization technique available. Here I think you'll find some very interesting things about the models we're using.<\/span>\r\n\r\nKeep in mind - net1 <\/strong>detects bear or not-bear, and net2<\/strong> detects bear, cow or sheep.<\/span>\r\n
    <\/h6>\r\nLet's set up for Grad-CAM.<\/span>\r\n
    <\/h6>\r\n
    %create a layer graph from the network, remove its final classification layer.<\/span>\r\n\r\nlgraph = layerGraph(net2);\r\nlgraph = removeLayers(lgraph, lgraph.Layers(end).Name);\r\ndlnet = dlnetwork(lgraph);\r\n\r\n% you need to know the probability layer and a feature layer for<\/span>\r\n% inceptionv3, using softmax (315) and the last ReLU layer<\/span>\r\nsoftmaxName = 'predictions_softmax';\r\nfeatureLayerName = 'activation_94_relu';<\/pre>\r\n
    <\/h6>\r\n
    [classfn,score] = classify(net1,img); %bear1<\/span>\r\nimshow(img);\r\ntitle(sprintf(\"%s (%.2f)\", classfn, score(classfn)));\r\n\r\ndlImg = dlarray(single(img),'SSC'); \r\n[featureMap, dScoresdMap] = dlfeval(@gradcam, dlnet, dlImg, softmaxName, featureLayerName, classfn);\r\n\r\n% a few more lines for visualization<\/span>\r\ngradcamMap = sum(featureMap .* sum(dScoresdMap, [1 2]), 3);\r\ngradcamMap = extractdata(gradcamMap);\r\ngradcamMap = rescale(gradcamMap);\r\ngradcamMap = imresize(gradcamMap, inputSize, 'Method', 'bicubic');\r\n<\/pre>\r\nRunning this code for bears 1,2 and 3, using the bear, not-bear model, here are the results:<\/span>\r\n\r\n\r\n\r\n
    \"\"<\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nNothing too exciting with the images above. Those are definitely bears, and the visualization is more or less targeting the bear.<\/span>\r\n
    <\/h6>\r\nNOW! Let's use the other model. [Same code as above, just replacing net1 with net2] Remember at the very beginning: The network predicted all bears correctly as bears. So why spend time to visualize correct results?<\/span>\r\n\r\n\r\n\r\n
    \"\"<\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nDo you see it? This model is always activating on the lower left corner! So the model is predicting correctly, but why<\/em> it predicted a bear is suspect.<\/span>\r\n
    <\/h6>\r\nOne more thing before we move on: what if we just crop out the strongest\/incorrect features the network is focused on?<\/span>\r\n\"\"\r\n
    <\/h6>\r\n... interesting! Removing the strongest features in the image caused the network to predict incorrectly. This means the network is not likely learning real animal features, and focused on other aspects of the images. Most often I find that visualizations are great to gain insight into a model, but rarely have I been able prove an \"accurate\" model is not really predicting accurately at all!! My work here is done.<\/span>\r\n
    <\/h6>\r\n
    Occlusion Sensitivity<\/h2>\r\n
    <\/h6>\r\nThe final output visualization we'll cover today is occlusion sensitivity. This is by far the easiest to implement code-wise. <\/span>\r\n
    label = classify(net1,img);\r\n\r\nscoreMap = occlusionSensitivity(net1,img,label);\r\n\r\nfigure\r\nimshow(img)\r\nhold on\r\nimagesc(scoreMap,'AlphaData',0.5);\r\ncolormap jet<\/pre>\r\n\"\"\r\n
    <\/h6>\r\nHere you can see that the reason the sheep is labeled \"not bear\" is in the center of the animal. I was semi-disappointed to not have those big horns light up, but what can you do?<\/span>\r\n
    <\/h6>\r\n
    Gradient Attribution Techniques<\/h2>\r\nJust another note that there are more visualization techniques, such as Gradient Attribution Techniques, but I think we covered a lot already. To learn more, you can check this doc example<\/a>.<\/span>\r\n\r\n\r\n\r\n
    \"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\n
    Inside the Network Insights<\/h1>\r\n
    <\/h6>\r\n
    Deep Dream<\/h2>\r\n
    <\/h6>\r\nHere we can visualize the network's learned features. Let's visualize the first 25 features learned by the network at the second convolutional layer.<\/span>\r\n
    <\/h6>\r\n\r\n\r\n\r\n
    \r\n
    <\/h6>\r\n
    layer = 'conv2d_2';\r\nchannels = 1:25;\r\n\r\nI = deepDreamImage(net2,layer,channels, ...\r\n    'PyramidLevels',1, ...\r\n    'Verbose',0);\r\n\r\nfigure\r\nfor i = 1:25\r\n    subplot(5,5,i)\r\n    imshow(I(:,:,:,i))\r\nend\r\n<\/pre>\r\n<\/td>\r\n
    		\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\nI'll be honest, while I find deep dream visually pleasing, I don't tend to use this very often as a debugging technique, though I'd be interested to hear from anyone who has an example of success. Sometimes, deep dream can be helpful for later layers of the network, where other techniques will certainly fail.<\/span>\r\n
    <\/h6>\r\n \r\n
    <\/h6>\r\nHowever, in this particular example, I wasn't able to find anything particularly appealing even after running deep dream for 100 iterations.<\/span>\r\n
    iterations = 100;\r\nlayerName = 'new_fc';\r\n\r\nI = deepDreamImage(net1,layerName,1, ...\r\n    'Verbose',false, ...\r\n    'NumIterations',iterations);\r\n\r\nfigure\r\nimshow(I)<\/pre>\r\n
    \"\"<\/h6>\r\nThis image is visualizing the \"bear\" class. I'm not seeing particularly insightful to comment on, though I have seen very attractive deep dream images created from other pretrained networks in this example<\/u><\/a>.<\/span>\r\n
    <\/h6>\r\n
    Activations<\/h2>\r\n
    <\/h6>\r\nSimilar to deep dream, you can use activations to visualize the input image after it passes through specific channels, which are showing the learned features from the network.<\/span>\r\n
    im = imread('bear2.JPG');\r\n\r\nimgSize = size(im);\r\nimgSize = imgSize(1:2);\r\n\r\nact1 = activations(net1,im,'conv2d_7');\r\n\r\nsz = size(act1);\r\nact1 = reshape(act1,[sz(1) sz(2) 1 sz(3)]);\r\n\r\nI = imtile(mat2gray(act1),'GridSize',[7 7]);\r\nimshow(I)\r\n<\/pre>\r\n\"\"\r\n
    <\/h6>\r\nThen, you can use activations to quickly pull the strongest channel activating.<\/span>\r\n
    <\/h6>\r\n
    % find the strongest channel<\/span>\r\n[maxValue,maxValueIndex] = max(max(max(act1)));\r\nact1chMax = act1(:,:,:,maxValueIndex);\r\nact1chMax = mat2gray(act1chMax);\r\nact1chMax = imresize(act1chMax,imgSize);\r\n\r\nI = imtile({im,act1chMax});\r\nimshow(I)\r\n<\/pre>\r\n\"\"\r\n
    <\/h6>\r\nSee a related documentation example<\/a> for more ways to use activations.<\/span>\r\n
    <\/h6>\r\n
    TSNE<\/h2>\r\nMaria wrote a blog post<\/a> about this a while back, and I'm happy to report a new example is in documentation here<\/strong><\/a>. This can help show similarities and differences between classes.<\/span>\r\n
    <\/h6>\r\n\r\n
    <\/h6>\r\n
    <\/h6>\r\nAny other visualizations you like that I should add to this collection? Let me know in the comments below!<\/span>\r\n
    <\/h6>\r\n
    Follow @jo_pings<\/a>