{"id":912,"date":"2019-01-18T06:00:36","date_gmt":"2019-01-18T06:00:36","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=912"},"modified":"2021-04-06T15:51:14","modified_gmt":"2021-04-06T19:51:14","slug":"neural-network-feature-visualization","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2019\/01\/18\/neural-network-feature-visualization\/","title":{"rendered":"Neural Network Feature Visualization"},"content":{"rendered":"

Visualization of the data and the semantic content learned by a network<\/h4>\r\nThis post comes from Maria Duarte Rosa, who is going to talk about different ways to visualize features learned by networks.<\/span>\r\n
<\/h6>\r\nToday, we'll look at two ways to gain insight into a network using two methods:\u00a0k-nearest neighbors<\/strong> and t-SNE<\/strong>, which we'll describe in detail below.\r\n
<\/h6>\r\n
\"Semantic

Visualization of a trained network using t-SNE<\/p><\/div>\r\n

<\/h6>\r\n

Dataset and Model<\/h3>\r\n

<\/h6>\r\nFor both of these exercises, we'll be using ResNet-18<\/a>,\u00a0and our favorite food dataset, which you can download here<\/a>. (Please be aware this is a very large download. We're using this for examples purposes only, since food is relevant to everyone! This code should work with any other dataset you wish).\r\n
<\/h6>\r\nThe network has been retrained<\/a> to identify the 5 categories of objects from the data:\r\n
<\/h6>\r\n\r\n\r\n\r\n\r\n
Salad<\/strong><\/td>\r\nPizza<\/strong><\/td>\r\nFries<\/strong><\/td>\r\nBurger<\/strong><\/td>\r\nSushi<\/strong><\/td>\r\n<\/tr>\r\n
\"\"<\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\nNext we want to visualize our network and understand features used by a neural network to classify data. The following are two ways to visualize high-level features of a network, to gain insight into a network beyond accuracy.\r\n
<\/h6>\r\n
<\/h6>\r\n

k-nearest neighbors search<\/h3>\r\nA nearest neighbor search is a type of optimization problem where the goal is to find the closest (or most similar) points in space to a given point.\u00a0 K-nearest neighbors search identifies the top k closest neighbors to a point in feature space. Closeness in metric spaces is generally defined using a distance metric such as the Euclidean distance or Minkowski distance. The more similar the points are, the smaller this distance should be. This technique is often used as a machine learning classification method, but can also be used for visualization of data and high-level features of a neural network, which is what we're going to do.\r\n

<\/h6>\r\nLet's start with 5 test images from the food dataset:\r\n
<\/h6>\r\n
idxTest = [394 97 996 460 737];\r\nim = imtile(string({imdsTest.Files{idxTest}}),'ThumbnailSize',[100 100], 'GridSize', [5 1]);<\/pre>\r\n
\"\"<\/h6>\r\n
<\/h6>\r\nand look for the 10 nearest neighbors of these images in the training data in the pixel space. The code below is going to\u00a0get the features (i.e. \"activations\") for all test images, and find which ones are element-wise closest to our chosen sample images.\r\n
<\/h6>\r\nGet the features, aka activations\r\n
<\/h6>\r\n
dataTrainFS = activations(netFood, imdsTrainAu, 'data', 'OutputAs', 'rows');\r\nimgFeatSpaceTest = activations(netFood, imdsTestAu,'data', 'OutputAs', 'rows');\r\ndataTestFS = imgFeatSpaceTest(idxTest,:);\r\n<\/pre>\r\nCreate KNN model and search for nearest neighbours\r\n
Mdl = createns(dataTrainFS,'Distance','euclidean');\r\nidxKnn = knnsearch(Mdl,dataTestFS, 'k', 10);\r\n<\/pre>\r\n\"\"\r\n\r\n\r\nSearching for similarities in pixel space does not generally return any meaningful information about the semantic content of the image but only similarities in pixel intensity and color distribution. The 10 nearest neighbors in the data (pixel) space do not necessarily correspond to the same class as the test image. There is no \"learning\" taking place.\r\n
<\/h6>\r\nTake a look at the 4th row:\r\n
\"\"<\/h6>\r\n
<\/h6>\r\nThe image of the fries is yellow and brighter at the top, and dark at the bottom.\u00a0Most of the nearest neighbors in pixel space seem to be images of any class that contains the same pixel intensity and color pattern (they are somewhat brighter at the top and dark at the bottom).\r\n\r\nLet's compare this with images passed through the network and search for the 10 nearest neighbors in feature space,\u00a0where the features are the output of the final average pooling layer of the network, pool5.\r\n
dataTrainFS = activations(netFood, imdsTrainAu, 'pool5', 'OutputAs', 'rows');\r\nimgFeatSpaceTest = activations(netFood, imdsTestAu,'pool5', 'OutputAs', 'rows');\r\ndataTestFS = imgFeatSpaceTest(idxTest,:);\r\n<\/pre>\r\nCreate KNN model and search for nearest neighbours\r\n
Mdl = createns(dataTrainFS,'Distance','euclidean');\r\nidxKnn(:,:) = knnsearch(Mdl,dataTestFS, 'k', 10);<\/pre>\r\n
<\/h6>\r\n
\"\"
The first column (highlighted) is the test image, the remaining columns are the 10 nearest neighbors<\/p><\/div>\r\n\r\nNow we can see the color and intensity no longer matter, but rather the higher level features of the objects in the image. The nearest neighbors are now images of the same class. These results show that the features from the deep neural network contain information about the semantic content of the images. In other words, the network learned to discriminate between classes by learning high-level object specific features similarly to what allows humans to distinguish hamburgers from pizzas or Caesar salads from sushi.\r\n
<\/h6>\r\n
K-NN: What can we learn from this?<\/h5>\r\nThis can confirm what we expect to see from the network, or simply another visualization of the network in a new way. If the training accuracy of the network is high but the nearest neighbors in feature space (assuming the features are the output of one of the final layers of the network) are not objects from the same class, this may indicate that the network has not captured any semantic knowledge related to the classes but might have learned to classify based on some artifact of the training data.\r\n
<\/h6>\r\n
<\/h6>\r\n \r\n
Semantic clustering with t-SNE<\/h3>\r\nt-Distributed Stochastic Neighbor Embedding (t-SNE) is a non-linear dimensionality reduction technique that allows embedding high-dimensional data in a lower-dimensional space. (Typically we choose the lower dimensional space to be two or three dimensions, since this makes it easy to plot and visualize). This lower dimensional space is estimated in such a way that it preserves similarities from the high dimensional space. In other words, two similar objects have high probability of being nearby in the lower dimensional space, while two dissimilar objects should be represented by distant points. <\/strong> This technique can be used to visualize deep neural network features.\r\n
<\/h6>\r\nLet's apply this technique to the training images of the dataset and get a two dimensional and three dimensional embedding of the data.\r\n
<\/h6>\r\nSimilar to k-nn example, we'll start by visualizing the original data (pixel space) and the output of the final averaging pooling layer.\r\n
layers = {'data', 'pool5'};\r\nfor k = 1:length(layers)\r\n   dataTrainFS = activations(netFood, imdsTrainAu, layers{k}, 'OutputAs', 'rows');\r\n   AlltSNE2dim(:,:,k) = tsne(dataTrainFS);\r\n   AlltSNE3dim(:,:,k) = tsne(dataTrainFS), 'NumDimensions', 3);\r\nend\r\n\r\nfigure;\r\nsubplot(1,2,1);gscatter(AlltSNE2dim(:,1,1), AlltSNE2dim(:,2,1), labels);\r\ntitle(sprintf('Semantic clustering - %s layer', layers{1}));\r\nsubplot(1,2,2);gscatter(AlltSNE2dim(:,1,end), AlltSNE2dim(:,2,end), labels);\r\ntitle(sprintf('Semantic clustering - %s layer', layers{end}));\r\nfigure;\r\nsubplot(1,2,1);scatter3(AlltSNE3dim(:,1,1),AlltSNE3dim(:,2,1),AlltSNE3dim(:,3,1), 20*ones(3500,1),  labels)\r\ntitle(sprintf('Semantic clustering - %s layer', layers{1}));\r\nsubplot(1,2,2);scatter3(AlltSNE3dim(:,1,end),AlltSNE3dim(:,2,end),AlltSNE3dim(:,3,end), 20*ones(3500,1),  labels)\r\ntitle(sprintf('Semantic clustering - %s layer', layers{end}));<\/pre>\r\n
\"\" \"\"<\/h6>\r\n
<\/h6>\r\nBoth in the two and three dimensional images, it is possible to see that the data is scattered all over the space - in a very random pattern. But when we plot the embedding for the output of 'pool5' the pattern is very different. Now we can clearly see clusters of points according to the semantic content of the image. The clusters correspond to the 5 different classes available in the data. This means that the high-level<\/em> representations learned by the network contain discriminative information about the objects in the images, which allows the network to accurately predict the class of the object.\r\n
<\/h6>\r\nIn addition to the information that these visualizations provide about the network, they can also be useful to inspect the data itself. For example, let's visualize a few images where the images are in the wrong cluster, and see if we can get some insight into why the network miss-predicted the output.\r\n
<\/h6>\r\n
Examples of images in the wrong semantic cluster<\/h3>\r\n
<\/h6>\r\nLet's take a closer look at the 2D image of the pool5 layer, and zoom in on a few of the misclassified images.\r\n
<\/h6>\r\n\r\n\r\n\r\n\r\n
\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n\r\n
<\/h6>\r\n
<\/h6>\r\n
im = imread(imdsTrain.Files{1619});\r\nfigure;imshow(im);title('Hamburger that looks like a salad');<\/pre>\r\n\"\"\r\n
<\/h6>\r\nA hamburger in the salad cluster. Unlike other hamburger images, there is a significant amount of salad in the photo and no bread\/bun.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\n
im = imread(imdsTrain.Files{125});\r\nfigure;imshow(im);title('Ceaser salad that looks like a hamburger');<\/pre>\r\n.\"\"\r\n
<\/h6>\r\nA salad in the hamburger cluster. This may be because the image contains a bun or bread in the background.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\n
im = imread(imdsTrain.Files{3000});\r\nfigure;imshow(im);title('Sushi that looks like a hamburger');<\/pre>\r\n
\u00a0\u00a0\u00a0<\/span>\u00a0<\/span>\"\"<\/h6>\r\n \r\n
<\/h6>\r\nMaybe because it has some features that look like something one could find in a burger?\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nFinally, I think it is interesting to visualize the t-SNE for all the layers of the network, where we can see the data starts as random points, and slowly becomes clustered appropriately.\r\n
<\/h6>\r\n
\"\"<\/h6>\r\n
<\/h6>\r\nYou can download the code using the small \"Get the MATLAB code\" link below. You'll need to bring your own pretrained network and dataset, since that is not included.<\/span>\r\n
<\/h6>\r\nHopefully you find these visualizations interesting and useful! Have a question for Maria? Leave a comment below!<\/span>\r\n
<\/h6>\r\n\r\n