If you are \u00a0interested in learning more about deep learning or trying out some of latest deep learning research in MATLAB this blog post will walk you through the first steps to getting started. The entry on File Exchange provides everything you need to download one of the most popular deep neural networks and use it to classify images using\u00a0live video from a webcam. Before you read through the rest of this post, I would highly recommend you watch this video<\/a><\/strong> <\/span>by Joe Hicklin (pictured below) that illustrates what I will explain in more detail.<\/p>\n <\/p>\n This post covers\u00a0how to download a pre-trained deep convolutional neural network and use it to classify images in a live video stream.<\/p>\n The pre-trained model I’ll use in this post is known as AlexNet. This model was\u00a0trained to recognize 1000 different categories of objects in images, and was trained using over a million images. The AlexNet model<\/a> is available as a support package in MATLAB, you can learn more about the AlexNet support package from this blog post<\/a>. Now let’s load the pre-trained network into MATLAB.<\/p>\n Now let us take a look at the structure of AlexNet model. Notice that the first layer requires a 227×227 RGB image, and the last layer classifies 1000 different objects.<\/p>\n Before we connect a webcam and try classifying images on a live stream, lets try classifying a single test image.<\/p>\n <\/p>\n To classify the input image we use the ‘classify’ function.\u00a0Since AlexNet expects the input image to be 227×227 let’s resize the<\/span> input image.<\/span><\/p>\n You’ll see that the network accurately classifies the image as a bell <\/span>pepper. You can also see the prediction score or confidence of the <\/span>prediction for each of the 1000 classes by outputing the score from the <\/span>classify method.<\/span><\/p>\n Now that we’ve tried the deep learning image classifier on a single image lets try doing the same with a live video stream. We connect to a webcam using the webcam object in MATLAB.<\/p>\n To learn more about deep learning in MATLAB take a look at this page.<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" \nAvi’s pick of the week is Deep Learning in 11 Lines of MATLAB Code by the MathWorks Deep Learning Toolbox Team. This post is follow up to this post by Jiro and provides a more detailed… read more >><\/a><\/p>\n","protected":false},"author":132,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[16],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/8432"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/users\/132"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/comments?post=8432"}],"version-history":[{"count":26,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/8432\/revisions"}],"predecessor-version":[{"id":10133,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/posts\/8432\/revisions\/10133"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/media?parent=8432"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/categories?post=8432"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/pick\/wp-json\/wp\/v2\/tags?post=8432"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/blogs.mathworks.com\/pick\/files\/Joe.png\")
<\/a><\/p>\nLoad Pre-Trained Network<\/h2>\n
nnet = alexnet; % Will prompt support package install if unavaliable<\/span>\r\n<\/pre>\nView Network Structure<\/h2>\n
nnet.Layers\r\n<\/pre>\n
ans = \r\n\r\n 25x1 Layer array with layers:\r\n\r\n 1 'data' Image Input 227x227x3 images with 'zerocenter' normalization\r\n 2 'conv1' Convolution 96 11x11x3 convolutions with stride [4 4] and padding [0 0]\r\n 3 'relu1' ReLU ReLU\r\n 4 'norm1' Cross Channel Normalization cross channel normalization with 5 channels per element\r\n 5 'pool1' Max Pooling 3x3 max pooling with stride [2 2] and padding [0 0]\r\n 6 'conv2' Convolution 256 5x5x48 convolutions with stride [1 1] and padding [2 2]\r\n 7 'relu2' ReLU ReLU\r\n 8 'norm2' Cross Channel Normalization cross channel normalization with 5 channels per element\r\n 9 'pool2' Max Pooling 3x3 max pooling with stride [2 2] and padding [0 0]\r\n 10 'conv3' Convolution 384 3x3x256 convolutions with stride [1 1] and padding [1 1]\r\n 11 'relu3' ReLU ReLU\r\n 12 'conv4' Convolution 384 3x3x192 convolutions with stride [1 1] and padding [1 1]\r\n 13 'relu4' ReLU ReLU\r\n 14 'conv5' Convolution 256 3x3x192 convolutions with stride [1 1] and padding [1 1]\r\n 15 'relu5' ReLU ReLU\r\n 16 'pool5' Max Pooling 3x3 max pooling with stride [2 2] and padding [0 0]\r\n 17 'fc6' Fully Connected 4096 fully connected layer\r\n 18 'relu6' ReLU ReLU\r\n 19 'drop6' Dropout 50% dropout\r\n 20 'fc7' Fully Connected 4096 fully connected layer\r\n 21 'relu7' ReLU ReLU\r\n 22 'drop7' Dropout 50% dropout\r\n 23 'fc8' Fully Connected 1000 fully connected layer\r\n 24 'prob' Softmax softmax\r\n 25 'output' Classification Output crossentropyex with 'tench', 'goldfish', and 998 other classes\r\n<\/pre>\n
Classify Test Images<\/h2>\n
I = imread('peppers.png'<\/span>);\r\nimshow(I)\r\ntitle('Input Image'<\/span>)\r\n<\/pre>\n![\"\"](\"https:\/\/blogs.mathworks.com\/pick\/files\/DeepLearningIn11LinesPost_01.png\")
<\/a><\/p>\nIin = imresize(I,[227 227]);\r\nfigure\r\nlabel = classify(nnet, Iin); % Classify the picture<\/span>\r\nimshow(Iin) % Show the picture<\/span>\r\ntitle(char(label)) % Show the label<\/span>\r\n<\/pre>\n
\n<\/a> ![\"\"](\"https:\/\/blogs.mathworks.com\/pick\/files\/DeepLearningIn11LinesPost_02.png\")
<\/a><\/p>\n<\/h2>\n
Try on Live Video<\/h2>\n
camera = webcam; % Connect to camera<\/span>\r\npicture = snapshot(camera); % Take picture<\/span>\r\nimshow(picture) % Show picture<\/span>\r\n\r\n% Run live video capture and classification in loop<\/span>\r\nfor<\/span> n = 1:100\r\n picture = snapshot(camera); % Take picture<\/span>\r\n picture = imresize(picture,[227,227]); % Resize picture<\/span>\r\n label = classify(nnet, picture); % Classify picture<\/span>\r\n imshow(picture); % Show picture<\/span>\r\n title(char(label)) % Show label<\/span>\r\n drawnow\r\nend<\/span><\/pre>\n![\"\"](\"https:\/\/blogs.mathworks.com\/pick\/files\/DeepLearningIn11LinesPost_03.png\")
<\/p>\n<\/div>\n
![\n<\/a>](\"https:\/\/blogs.mathworks.com\/pick\/files\/DeepLearningIn11LinesPost.png\"){kind=link}