{"id":1432,"date":"2019-03-13T15:37:51","date_gmt":"2019-03-13T15:37:51","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=1432"},"modified":"2019-06-12T17:01:50","modified_gmt":"2019-06-12T17:01:50","slug":"gtc-here-we-come","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2019\/03\/13\/gtc-here-we-come\/","title":{"rendered":"MATLAB Demos at GTC: Style Transfer and Celebrity Lookalikes"},"content":{"rendered":"This post is all about NVIDIA and their upcoming <a href=\"https:\/\/www.nvidia.com\/en-us\/gtc\/\">GPU Technology Conference<\/a> commonly referred to as GTC. Thousands of people attend every year at GTCs worldwide. GTC San Jose boasts around 9000 attendants. If you're one of those people, stop by and say hello! There's going to be many presentations, including a few from MathWorks on <a href=\"https:\/\/www.mathworks.com\/products\/gpu-coder.html\">GPU Coder<\/a>, <a href=\"https:\/\/gputechconf2019.smarteventscloud.com\/connect\/search.ww#loadSearch-searchPhrase=S9469&searchType=session&tc=0&sortBy=dayTime&p=\">MATLAB and NVIDIA Docker,<\/a> and a hands-on MATLAB workshop: <a href=\"https:\/\/gputechconf2019.smarteventscloud.com\/connect\/search.ww#loadSearch-searchPhrase=MATLAB&searchType=session&tc=0&sortBy=dayTime&p=\">See the schedule here<\/a>.\r\n<h6><\/h6>\r\n<h2>What are we going to show?<\/h2>\r\n<h6><\/h6>\r\n\r\n<h3>Deep Learning on the Cloud<\/h3>\r\nEven if you're not going to GTC, we want you to know that we support NVIDIA GPU Cloud Resources, through their <a href=\"https:\/\/ngc.nvidia.com\/catalog\/containers\">NGC platform.<\/a> This means you can run MATLAB with NVIDIA GPUs in the cloud. A new (NVIDIA!) blog post explains the details with an example: <a href=\"https:\/\/devblogs.nvidia.com\/speeding-up-semantic-segmentation-matlab-nvidia-ngc\/\">https:\/\/devblogs.nvidia.com\/<\/a>\r\n\r\n<h6><\/h6>\r\nMore links on cloud resources:\r\n<ul>\r\n \t<li>Here's a super detailed video to get you set up using GPUs on the cloud: <a href=\"https:\/\/www.mathworks.com\/videos\/setting-up-the-matlab-deep-learning-container-on-ngc-1537515024196.html\">link to video<\/a><\/li>\r\n \t<li>and more details can be found here: <a href=\"https:\/\/www.mathworks.com\/help\/cloudcenter\/matlab-on-the-cloud.html\">link to documentation<\/a><\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<h3>Demos<\/h3>\r\n\r\nAt the show, we'll be luring people to our booth with a raffle for a free MATLAB Home license, and cool demos including:\r\n<ul>\r\n \t<li style=\"list-style-type: none\">\r\n\r\n \t<li>Deep Learning on DGX<\/li>\r\n \t<li>Deep Learning running TensorRT<\/li>\r\n \t<li>Age Detection using YOLO on a Jetson Xavier board<\/li>\r\n<\/ul>\r\n\r\n\r\nAnd your choice between two deep learning demos I just completed: \"Style Transfer\" and \"Celebrity Doppelganger.\" The code for Celebrity Doppelganger can be downloaded below. Just add your own images, and you're good to go!\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<strong>Style Transfer<\/strong>\r\n\r\n<h6><\/h6>\r\nThis demo uses deep learning to create images in the style of another image (such as Van Gogh, or Monet). A network was trained to extract specific features from an image at many points in the network. The network will encode and decode features of a new image that will take the style from a reference image:\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"1598\" height=\"929\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/03\/styletransfer.png\" alt=\"\" class=\"alignnone size-full wp-image-1460\" \/>\r\n\r\n\r\n\r\n\r\n<h6><\/h6>\r\nThis isn't the zippiest of demos, (I've been averaging 0.5s per image with my setup) so we can't run this example streaming. Instead, I have it running on a timer to take a picture every few seconds and allow people to customize their own pictures in which ever style they'd like.\r\n<h6><\/h6>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" width=\"1600\" height=\"929\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/03\/styletransfer2.png\" alt=\"\" class=\"alignnone size-full wp-image-1462\" \/>\r\n\r\n<h6><\/h6>\r\n<strong>Celebrity Doppelganger <\/strong>\r\n<h6><\/h6>\r\nThis demo is extracting features from a network (I chose ResNet-101 and pulled features from the final fully connected layer) and running a K-Nearest Neighbor algorithm to determine the closest match of features belonging to a celebrity. \r\n<h6><\/h6>\r\n\r\nThe code to build the celebrity KD Tree Searcher looks like this:\r\n<pre>function MDL = buildCelebModel(imds_celeb,net,layer)\r\n% get celebrity activations\r\n\r\nnum_files = length(imds_celeb.Files);\r\nactivs_c = [];\r\n\r\nfor ii = 1:num_files\r\n    ipicture = imresize(imds_celeb.readimage(ii), [448,448]);\r\n    bbox = face_predict(ipicture);\r\n    if ~isempty(bbox)\r\n        bboxpt = bbox(1,:);\r\n        bufx = floor(.3*bboxpt(3));        bufy = floor(.3*bboxpt(4));\r\n        xs = max(bboxpt(1)-bufx, 1);        ys = max(bboxpt(2)-bufy, 1);\r\n        xe = min(bboxpt(1)+bboxpt(3)-1+bufx, size(ipicture,2));\r\n        ye = min(bboxpt(2)+bboxpt(4)-1+bufy, size(ipicture,1));\r\n       \r\n        face = ipicture(ys:ye, xs:xe, :);\r\n        face = imresize(face, [224,224]);\r\n    end\r\n    \r\n    xx = activations(net,single(face),layer);\r\n    activs_c = vertcat(activs_c,squeeze(xx(1,1,:))'); %'\r\nend\r\nMDL = KDTreeSearcher(activs_c);\r\nend<\/pre>\r\n\r\n<h6><\/h6>\r\nThe idea is to pull activations from each face, and then build a model (MDL) that you can use to find the closest match. When I run this through my webcam, extract a face from the camera, I can run this code:\r\n<h6><\/h6>\r\n<pre>\r\nactivs_u = squeeze(xx(1,1,:))';%'\r\n[Idx,D] = knnsearch(app.mdl,activs_u,'K',num);\r\n<\/pre>\r\n<h6><\/h6>\r\nAnd I will get the Index (Idx) corresponding to the image number of my closest match. Inside the app, it will look something like this:\r\n<h6><\/h6>\r\n\r\n<img decoding=\"async\" loading=\"lazy\" width=\"661\" height=\"548\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/03\/doppel.png\" alt=\"\" class=\"alignnone size-full wp-image-1464\" \/>\r\n\r\n\r\n\r\n<h6><\/h6>\r\n<em>Whether or not you believe I look like Uma I guess is up for debate, but let's just say I was happy with my match!!<\/em>\r\n\r\nUpdate: when I ran it again the following day, apparently I look like Padma? This algorithm is suspicious!\r\n<img decoding=\"async\" loading=\"lazy\" width=\"659\" height=\"546\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/03\/doppel2.png\" alt=\"\" class=\"alignnone size-full wp-image-1466\" \/>\r\n<h6><\/h6>\r\nAlso keep in mind at a conference such as this, it's not really about showing the most ground breaking or highest accuracy algorithm. It's about letting people interact with deep learning and hopefully starting a conversation.\r\n<h6><\/h6>\r\n\r\nThat said, I still have a few days to play with these demos before I leave for San Jose. <strong>Any suggestions on how to make these demos better before the conference?<\/strong>\r\n<h6><\/h6>\r\n\r\nI put the code for the doppelganger example in the download code button below. Let me know what you think of the demos, and please stop by the booth if you're going to be at GTC San Jose!\r\n\r\n<h6><\/h6>\r\n<p><a href=\"https:\/\/twitter.com\/jo_pings?ref_src=twsrc%5Etfw\" class=\"twitter-follow-button\" data-size=\"large\" data-show-count=\"false\">Follow @jo_pings<\/a><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\r\n\r\n<script language=\"JavaScript\"> <!-- \r\n    function grabCode_a710f144b80042c592b9fe35aab1fc59() {\r\n        \/\/ Remember the title so we can use it in the new page\r\n        title = document.title;\r\n\r\n        \/\/ Break up these strings so that their presence\r\n        \/\/ in the Javascript doesn't mess up the search for\r\n        \/\/ the MATLAB code.\r\n        t1='a710f144b80042c592b9fe35aab1fc59 ' + '##### ' + 'SOURCE BEGIN' + ' #####';\r\n        t2='##### ' + 'SOURCE END' + ' #####' + ' a710f144b80042c592b9fe35aab1fc59';\r\n    \r\n        b=document.getElementsByTagName('body')[0];\r\n        i1=b.innerHTML.indexOf(t1)+t1.length;\r\n        i2=b.innerHTML.indexOf(t2);\r\n \r\n        code_string = b.innerHTML.substring(i1, i2);\r\n        code_string = code_string.replace(\/REPLACE_WITH_DASH_DASH\/g,'--');\r\n\r\n        \/\/ Use \/x3C\/g instead of the less-than character to avoid errors \r\n        \/\/ in the XML parser.\r\n        \/\/ Use '\\x26#60;' instead of '<' so that the XML parser\r\n        \/\/ doesn't go ahead and substitute the less-than character. \r\n        code_string = code_string.replace(\/\\x3C\/g, '\\x26#60;');\r\n\r\n        copyright = 'Copyright 2018 The MathWorks, Inc.';\r\n\r\n        w = window.open();\r\n        d = w.document;\r\n        d.write('<pre>\\n');\r\n        d.write(code_string);\r\n\r\n        \/\/ Add copyright line at the bottom if specified.\r\n        if (copyright.length > 0) {\r\n            d.writeln('');\r\n            d.writeln('%%');\r\n            if (copyright.length > 0) {\r\n                d.writeln('% _' + copyright + '_');\r\n            }\r\n        }\r\n\r\n        d.write('<\/pre>\\n');\r\n\r\n        d.title = title + ' (MATLAB code)';\r\n        d.close();\r\n    }   \r\n     --> <\/script><p style=\"text-align: right; font-size: xx-small; font-weight:lighter;   font-style: italic; color: gray\">Copyright 2018 The MathWorks, Inc.<br><a href=\"javascript:grabCode_a710f144b80042c592b9fe35aab1fc59()\"><span class=\"get_ml_code\">Get the MATLAB code <noscript>(requires JavaScript)<\/noscript><\/span><\/a><br><br>\r\n      <br><\/p><!--\r\na710f144b80042c592b9fe35aab1fc59 ##### SOURCE BEGIN #####\r\n\r\n\r\n\r\n\r\n%% This example will build a celebrity KD Tree searcher Model, \r\n% and then find the closest match for a test image\r\n% you will need to have images in a 'celebs' folder and a 'FacesTest' for\r\n% this to work. \r\n\r\n% Step 1. get celebrity activations\r\n\r\nimds_celeb = imageDatastore('celebs\\');\r\nnum_files = length(imds_celeb.Files);\r\nactivs_c = [];\r\nnet = resnet101;\r\n%net = vgg16;\r\nlayer = 'fc1000';\r\nfor ii = 1:num_files\r\n    % face only\r\n    ipicture = imresize(imds_celeb.readimage(ii), [448,448]);\r\n    bbox = face_predict(ipicture);\r\n    \r\n    %\r\n    if ~isempty(bbox)\r\n        bboxpt = bbox(1,:);\r\n        bufx = floor(.3*bboxpt(3));\r\n        bufy = floor(.3*bboxpt(4));\r\n        xs = max(bboxpt(1)-bufx, 1);\r\n        ys = max(bboxpt(2)-bufy, 1);\r\n        xe = min(bboxpt(1)+bboxpt(3)-1+bufx, size(ipicture,2));\r\n        ye = min(bboxpt(2)+bboxpt(4)-1+bufy, size(ipicture,1));\r\n        \r\n        face = ipicture(ys:ye, xs:xe, :);\r\n        face = imresize(face, [224,224]);\r\n\r\n        \r\n    end\r\n\r\n    \r\n    xx = activations(net,single(face),layer);\r\n    activs_c = vertcat(activs_c,squeeze(xx(1,1,:))');\r\nend\r\nMdl = KDTreeSearcher(activs_c);\r\n%% Step 2 find face and predict on test image\r\n%%\r\ntest_images = imageDatastore('FacesTest');\r\ntest_image = test_images.readimage(1);\r\n\r\nipicture = imresize(test_image, [448,448]);\r\nbbox = face_predict(ipicture);\r\n\r\nif ~isempty(bbox)\r\n    bboxpt = bbox(1,:);\r\n    bufx = floor(.3*bboxpt(3));\r\n    bufy = floor(.3*bboxpt(4));\r\n    xs = max(bboxpt(1)-bufx, 1);\r\n    ys = max(bboxpt(2)-bufy, 1);\r\n    xe = min(bboxpt(1)+bboxpt(3)-1+bufx, size(ipicture,2));\r\n    ye = min(bboxpt(2)+bboxpt(4)-1+bufy, size(ipicture,1));\r\n    \r\n    face = ipicture(ys:ye, xs:xe, :);\r\n    face = imresize(face, [224,224]);\r\n    figure; imshow(face);\r\n    %\r\nend\r\n\r\nxx = activations(net,single(face),layer);\r\nactivs_u = squeeze(xx(1,1,:))';\r\n\r\n%% \r\n% Nearest neighbor\r\n\r\n[Idx,D] = knnsearch(Mdl,activs_u);\r\n\r\nimshow(imds_celeb.readimage(Idx));\r\ntitle(string(D));\r\n\r\n%%\r\n% Keep in mind, you don't have to use YOLO to predict location of faces.\r\n% You can use any face detection algorithm you'd like.\r\nfunction [selectedBbox] = face_predict(img_rz)\r\n%#codegen\r\n\r\n% This function detects the traffic signs in the image using Detection Network\r\n% (modified version of Yolo) and recognizes(classifies) using Recognition Network\r\n%\r\n% Inputs :\r\n%\r\n% im            : Input test image\r\n%\r\n% Outputs :\r\n%\r\n% selectedBbox  : Detected bounding boxes \r\n% idx           : Corresponding classes\r\n\r\n% Copyright 2017 The MathWorks, Inc.\r\n\r\ncoder.gpu.kernelfun;\r\n\r\n\r\n% Converting into BGR format\r\nimg_rz = img_rz(:,:,3:-1:1);\r\nimg_rz = im2single(img_rz);\r\n\r\n%% TSD\r\npersistent detectionnet;\r\nif isempty(detectionnet)   \r\n    detectionnet = coder.loadDeepLearningNetwork('yolo_face.mat','Detection');\r\nend\r\n\r\npredictions = detectionnet.predict(img_rz);\r\n\r\n\r\n%% Convert predictions to bounding box attributes\r\nclasses = 1;\r\nnum = 2;\r\nside = 11;\r\nthresh = 0.2;\r\n[h,w,~] = size(img_rz);\r\n\r\n\r\nboxes = single(zeros(0,4));    \r\nprobs = single(zeros(0,1));    \r\nfor i = 0:(side*side)-1\r\n    for n = 0:num-1\r\n        p_index = side*side*classes + i*num + n + 1;\r\n        scale = predictions(p_index);       \r\n        prob = zeros(1,classes+1);\r\n        for j = 0:classes\r\n            class_index = i*classes + 1;\r\n            tempProb = scale*predictions(class_index+j);\r\n            if tempProb > thresh\r\n                \r\n                row = floor(i \/ side);\r\n                col = mod(i,side);\r\n                \r\n                box_index = side*side*(classes + num) + (i*num + n)*4 + 1;\r\n                bxX = (predictions(box_index + 0) + col) \/ side;\r\n                bxY = (predictions(box_index + 1) + row) \/ side;\r\n                \r\n                bxW = (predictions(box_index + 2)^2);\r\n                bxH = (predictions(box_index + 3)^2);\r\n                \r\n                prob(j+1) = tempProb;\r\n                probs = [probs;tempProb];\r\n                                \r\n                boxX = (bxX-bxW\/2)*w+1;\r\n                boxY = (bxY-bxH\/2)*h+1;\r\n                boxW = bxW*w;\r\n                boxH = bxH*h;\r\n                boxes = [boxes; boxX,boxY,boxW,boxH];\r\n            end\r\n        end\r\n    end\r\nend\r\n\r\n%% Run Non-Maximal Suppression on the detected bounding boxess\r\ncoder.varsize('selectedBbox',[98, 4],[1 0]);\r\n[selectedBbox,~] = selectStrongestBbox(round(boxes),probs);\r\n\r\n##### SOURCE END ##### a710f144b80042c592b9fe35aab1fc59\r\n--><!-- AddThis Sharing Buttons below -->\r\n\r\n\r\n","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/03\/styletransfer.png\" onError=\"this.style.display ='none';\" \/><\/div><p>This post is all about NVIDIA and their upcoming GPU Technology Conference commonly referred to as GTC. Thousands of people attend every year at GTCs worldwide. GTC San Jose boasts around 9000... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2019\/03\/13\/gtc-here-we-come\/\">read more >><\/a><\/p>","protected":false},"author":156,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[7],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/1432"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/156"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=1432"}],"version-history":[{"count":28,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/1432\/revisions"}],"predecessor-version":[{"id":2389,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/1432\/revisions\/2389"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=1432"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=1432"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=1432"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}