{"id":4545,"date":"2020-06-01T11:57:48","date_gmt":"2020-06-01T15:57:48","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=4545"},"modified":"2021-04-06T15:48:18","modified_gmt":"2021-04-06T19:48:18","slug":"deep-learning-examples-r2020a-edition","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/06\/01\/deep-learning-examples-r2020a-edition\/","title":{"rendered":"Deep Learning Examples: R2020a Edition"},"content":{"rendered":"With two releases every year, you may find it challenging to keep up with the latest features.* In fact, some people who work here feel the same way! This release, I asked the Product Managers about the new features related to deep learning that they think you should know about in release 20a. Here are their responses:\r\n<h6><\/h6>\r\n&nbsp;\r\n<h2>Deep Learning<\/h2>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4631 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/covid-19-banner-update-words-2-1024x137.jpg\" alt=\"\" width=\"1024\" height=\"137\" \/>\r\n<h6><\/h6>\r\nStarting with Deep Learning Toolbox, there are three new features to get excited about in 20a.\r\n<h6><\/h6>\r\n<ol>\r\n \t<li><strong><u><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ref\/experimentmanager-app.html\">Experiment Manager<\/a><\/u><\/strong> (new) - A new app that keeps track all conditions when training neural networks. This can be extremely helpful to keep track of all training parameters, data and accuracy of each iteration of the network. More to come on this feature in future posts!<\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/gs\/get-started-with-deep-network-designer.html\"><strong><u>Deep Network Designer<\/u><\/strong><\/a> (updates) - Generate MATLAB code from the app, and train networks directly in the app.<\/li>\r\n \t<li><strong><u><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ref\/deepnetworkquantizer-app.html\">Post-Training Quantization<\/a><\/u><\/strong> (new) - This <a href=\"https:\/\/www.mathworks.com\/support\/search.html\/videos\/quantizing-a-deep-learning-network-in-matlab-1587620221222.html\">new video<\/a> describes the quantization workflow in MATLAB.<\/li>\r\n<\/ol>\r\n<h6><\/h6>\r\nThere are new examples which highlight these new features:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/train-conditional-generative-adversarial-network.html\">Train Conditional GANs<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/image-captioning-using-attention.html\">Train Image Captioning Networks using Attention<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/multilabel-text-classification-using-deep-learning.html\">Multilabel Text Classification Using Deep Learning<\/a>: This example shows how to classify text data that has multiple independent labels.<\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/compare-layer-weight-initializers-example.html\">Compare Layer Weight Initializers<\/a>: This example shows how to train deep learning networks with different weight initializers.<\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/train-network-with-multiple-outputs.html\">Train Network with Multiple Outputs<\/a>: This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<h2>Code Generation<\/h2>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4635 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/gpu-coder-banner-update-1024x136.jpg\" alt=\"\" width=\"1024\" height=\"136\" \/><\/h6>\r\n<strong>GPU Coder<\/strong>\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>Support for quite a few new networks, including SSD, bidirectional &amp; stateful LSTMs, DarkNet, Inception-ResNet-v2, NASNet-Large, and NASNet-Mobile<\/li>\r\n \t<li>New examples include:\r\n<ul>\r\n \t<li><strong><a href=\"https:\/\/www.mathworks.com\/help\/gpucoder\/examples\/gpu-object-detection-ssd.html\"><u>Code Generation for Object Detection by Using Single Shot Multibox Detector<\/u><\/a><\/strong> - This shows the code generation workflow for SSD network targeting cuDNN<\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/gpucoder\/examples\/code-generation-sequence-to-sequence-LSTM.html\"><strong><u>Code Generation for a Sequence-to-Sequence LSTM Network<\/u><\/strong><\/a> - This is an updated example which shows code generation for Stateful LSTM<\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<strong>MATLAB Coder<\/strong>\r\n<h6><\/h6>\r\n<ul>\r\n \t<li>Support for new networks including:\r\n<ul>\r\n \t<li>LSTM for ARM CPUs<\/li>\r\n \t<li>DarkNet-19, DarkNet-53, DenseNet-201, Inception-ResNet-v2, NASNet-Large, NASNet-Mobile, ResNet-18, and\u00a0Xception for Intel &amp; ARM CPUs<\/li>\r\n<\/ul>\r\n<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<h2>Signal and Audio<\/h2>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4639 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/deep-learning-for-signal-white-paper-banner2-1024x163.jpg\" alt=\"\" width=\"1024\" height=\"163\" \/><\/h6>\r\n<strong>Signal Processing<\/strong>\r\n<h6><\/h6>\r\n<ul>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\">New\u00a0standalone\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/signal\/ref\/signallabeler-app.html\">Signal Labeler\u00a0app<\/a> to label signal datasets.\u00a0You can see it in action in the\u00a0example\u00a0<a href=\"https:\/\/www.mathworks.com\/help\/signal\/ug\/label-signal-attributes-regions-of-interest-and-points.html\"><u><strong>Label Signal Attributes, Regions of Interest, and Points<\/strong><\/u><\/a><\/li>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" aria-setsize=\"-1\" data-aria-posinset=\"2\" data-aria-level=\"1\">New\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/signal\/ref\/signaldatastore.html\">signalDatastore<\/a>\u00a0to\u00a0train networks\u00a0with large collections of signals\u00a0or signal features across memory and disk. See how it can be used\u00a0with .MAT files\u00a0in the example\u00a0<a href=\"https:\/\/www.mathworks.com\/help\/signal\/examples\/waveform-segmentation-using-deep-learning.html\"><u><strong>Waveform Segmentation Using Deep Learning<\/strong><\/u><\/a><\/li>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" aria-setsize=\"-1\" data-aria-posinset=\"3\" data-aria-level=\"1\">Many additional signal processing\u00a0and wavelet\u00a0functions for feature extraction and time-frequency transformation like\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/signal\/ref\/spectrogram.html\">spectrogram<\/a>, <a href=\"http:\/\/www.mathworks.com\/help\/signal\/ref\/stft.html\">stft<\/a>, or\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/wavelet\/ref\/cwt.html\">cwt<\/a> now support gpuArrays for GPU acceleration, tall arrays for operating out of memory, and automatic CUDA code generation for running embedded GPUs. See for example\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/wavelet\/examples\/gpu-acceleration-scalograms-for-deep-learning.html\"><strong><u>GPU Acceleration of Scalograms for Deep Learning<\/u><\/strong><\/a><\/li>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" aria-setsize=\"-1\" data-aria-posinset=\"4\" data-aria-level=\"1\">Additional new\u00a0deep learning\u00a0examples including <a href=\"http:\/\/www.mathworks.com\/help\/signal\/examples\/iterative-labeling-for-time-domain-signals-using-deep-learning.html\"><strong><u>Iterative Approach for Creating Labeled Signal Sets with Reduced Human Effort<\/u><\/strong><\/a>, which\u202fuses a train-as-you-label iterative method for deep learning classifier training<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<strong>Audio Processing<\/strong>\r\n<h6><\/h6>\r\n<ul>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\">New example showing how to train and evaluate\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/audio\/examples\/train-gan-for-sound-synthesis.html\"><strong><u>GANs for generating synthetic audio<\/u><\/strong><\/a>.\u00a0This highlights the recently released API in Deep Learning Toolbox,\u00a0which includes\u00a0custom training loops<\/li>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" aria-setsize=\"-1\" data-aria-posinset=\"2\" data-aria-level=\"1\">New example discussing\u00a0the use of\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/audio\/examples\/speaker-verification-using-ivectors.html\"><strong><u>I-vectors\u00a0for Speaker Verification<\/u><\/strong><\/a>. I-vectors are a very popular modern\u00a0feature often used on audio signals. They are used with deep networks as well as with more traditional machine learning algorithms in lightweight\u00a0embedded systems<\/li>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" aria-setsize=\"-1\" data-aria-posinset=\"3\" data-aria-level=\"1\">New\u00a0<a href=\"http:\/\/www.mathworks.com\/help\/audio\/ref\/detectspeech.html\"><span style=\"font-family: courier;\">detectSpeech<\/span><\/a>\u00a0function to automatically detect\u00a0and annotate\u00a0regions of speech\u00a0in audio recordings<\/li>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" aria-setsize=\"-1\" data-aria-posinset=\"4\" data-aria-level=\"1\">New\u00a0<a href=\"https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/73326-text2speech\"><span style=\"font-family: courier;\">text2speech<\/span><\/a>\u00a0function\u00a0to\u00a0generate\u00a0pre-labeled synthetic speech\u00a0data\u00a0using web services, including Google's very popular Wavenet<\/li>\r\n<\/ul>\r\n<ul>\r\n \t<li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\">GPU acceleration for <a href=\"http:\/\/www.mathworks.com\/help\/audio\/ref\/mfcc.html\"><span style=\"font-family: courier;\">mfcc<\/span><\/a> and <a href=\"http:\/\/www.mathworks.com\/help\/audio\/ref\/melspectrogram.html\"><span style=\"font-family: courier;\">melSpectrogram<\/span><\/a>. <span style=\"font-family: courier;\">mfcc<\/span> now also supports generating CUDA code.<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<h2>Image Processing<\/h2>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4647 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/predictive-maintenance-ebook2-1024x162.jpg\" alt=\"\" width=\"1024\" height=\"162\" \/>\r\n<h6><\/h6>\r\nThere\u2019s a <a href=\"https:\/\/www.mathworks.com\/help\/images\/neural-style-transfer-using-deep-learning.html\"><strong><u>new style transfer demo<\/u><\/strong><\/a> available in Image Processing Toolbox.\u00a0 This demo will walk through the entire process of creating a network designed to take an image and transform it into the style of a reference image. \u00a0Now you can create images in the style of Picasso, van Gogh, or your favorite artist. The incorporation of custom training loops (<a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/03\/06\/advanced-deep-learning-key-terms\/\">Advanced Deep Learning: Key Terms)<\/a> makes techniques like style transfer relatively intuitive to implement.\r\n<h6><\/h6>\r\nFor Computer Vision, there is a new example describing how to <a href=\"https:\/\/www.mathworks.com\/help\/vision\/examples\/object-detection-using-single-shot-detector.html\"><strong><u>create a single shot detector<\/u><\/strong><\/a> (SSD).\r\n<h6><\/h6>\r\n<h2>Reinforcement Learning<\/h2>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4645 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/reinforcement-learning-ebook-part2-banner2-1-1024x175.jpg\" alt=\"\" width=\"1024\" height=\"175\" \/>\r\n<h6><\/h6>\r\n20a release of Reinforcement Learning Toolbox comes with a new agent, Twin Delayed Deep Deterministic Policy Gradient (TD3), additional support for continuous action spaces from existing agents (Policy Gradient, Actor Critic and Proximal Policy Optimization agent) and new examples that showcase how to build custom training algorithms and imitation learning.\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/reinforcement-learning\/ug\/train-ddpg-agent-with-pretrained-actor-network.html\"><strong>Train DDPG Agent with Pretrained Actor Network<\/strong><\/a>\r\n<h6><\/h6>\r\nReinforcement learning is a data hungry technique that requires many simulations for training. This example shows how to reduce training time, by initializing the neural network policy using existing data and supervised learning.<\/li>\r\n \t<li><strong><a href=\"https:\/\/www.mathworks.com\/help\/reinforcement-learning\/ug\/train-reinforcement-learning-policy-using-custom-training.html\">Train Reinforcement Learning Policy Using Custom Training Loop<\/a><\/strong>\r\n<h6><\/h6>\r\nWhile Reinforcement Learning Toolbox includes a variety of popular algorithms to train your system, you may want to customize these algorithms or create your own. This example shows the steps you need to follow to create a custom training algorithm with Reinforcement Learning Toolbox.<\/li>\r\n<\/ul>\r\n<h2>Radar &amp; Comms<\/h2>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4589 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/banner-machinelearning.jpg\" alt=\"\" width=\"1900\" height=\"260\" \/><\/h6>\r\n20a release is exciting for the Radar\/Comms area primarily because we have 4 new shipping examples. Here are the latest examples and features available in 20a:\r\n<h6><\/h6>\r\n<strong>RF Fingerprinting:<\/strong>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/comm\/examples\/design-a-deep-neural-network-with-simulated-data-to-detect-wlan-router-impersonation.html\">https:\/\/www.mathworks.com\/help\/comm\/examples\/design-a-deep-neural-network-with-simulated-data-to-detect-wlan-router-impersonation.html<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/comm\/examples\/test-a-deep-neural-network-with-captured-data-to-detect-wlan-router-impersonation.html\">https:\/\/www.mathworks.com\/help\/comm\/examples\/test-a-deep-neural-network-with-captured-data-to-detect-wlan-router-impersonation.html<\/a><\/li>\r\n<\/ul>\r\n<strong>5G Channel estimation:<\/strong>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/5g\/examples\/deep-learning-data-synthesis-for-5g-channel-estimation.html\">https:\/\/www.mathworks.com\/help\/5g\/examples\/deep-learning-data-synthesis-for-5g-channel-estimation.html<\/a><\/li>\r\n<\/ul>\r\n<strong>Log-likelihood estimation for comms channels<\/strong>\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/comm\/examples\/training-and-testing-a-neural-network-for-llr-estimation.html\">https:\/\/www.mathworks.com\/help\/comm\/examples\/training-and-testing-a-neural-network-for-llr-estimation.html<\/a><\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\nAccording to Product Manager, Rick Gentile, \"My personal favorites are the examples in links 1 and 2 because we have been getting so many requests for this type of application (RF Fingerprinting). It is a hot topic because it is used to prevent comms network spoofing. The 2<sup>nd<\/sup> example highlights the work we did with synthesized data using data we collected from a radio.\"\r\n<h6><\/h6>\r\nThat's it for this post. Hopefully we highlighted new features and examples you weren't aware of. What do you think of the list? Anything to add - leave a comment below!\r\n<h6><\/h6>\r\n<em>*You may also be aware that 20a was released in March, so I clearly find it challenging to keep up with the latest features! I've finally upgraded, and you should too!\u00a0<\/em>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/06\/covid-19-banner-update-words-2-1024x137.jpg\" onError=\"this.style.display ='none';\" \/><\/div><p>With two releases every year, you may find it challenging to keep up with the latest features.* In fact, some people who work here feel the same way! This release, I asked the Product Managers about... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/06\/01\/deep-learning-examples-r2020a-edition\/\">read more >><\/a><\/p>","protected":false},"author":156,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4545"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/156"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=4545"}],"version-history":[{"count":34,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4545\/revisions"}],"predecessor-version":[{"id":4661,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4545\/revisions\/4661"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=4545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=4545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=4545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}