{"id":2081,"date":"2019-05-13T15:45:15","date_gmt":"2019-05-13T15:45:15","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=2081"},"modified":"2021-04-06T15:50:46","modified_gmt":"2021-04-06T19:50:46","slug":"deep-learning-for-signal-processing-applications","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2019\/05\/13\/deep-learning-for-signal-processing-applications\/","title":{"rendered":"Deep Learning for Signal Processing Applications"},"content":{"rendered":"<em>I\u2019d like to introduce Frantz Bouchereau, development manager for Signal Processing Toolbox who is going to dive deep into insights on deep learning for signal processing, including the complete deep learning workflow for signal processing applications. It's a longer post than usual, but jam packed with actionable information. Enjoy!<\/em>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2265 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/deep-learning-for-signal-white-paper-banner_smaller.jpg\" alt=\"\" width=\"600\" height=\"150\" \/><\/h6>\r\n<span style=\"font-size: 16px;\"><strong>Introduction<\/strong><\/span>\r\n<h6><\/h6>\r\nToday we will highlight signal processing applications using deep learning techniques. Deep Learning developed and evolved for image processing and computer vision applications, but it is now increasingly and successfully used on signal and time series data.\r\n\r\n&nbsp;\r\n\r\nDeep learning is becoming popular in many industries including (but not limited to) the following areas:\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2171 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/image1_industry-1024x470.png\" alt=\"\" width=\"1024\" height=\"470\" \/><\/h6>\r\nThe unifying theme in these applications is that the data is not images but signals coming from different types of sensors like microphones, electrodes, radar, RF receivers, accelerometers, and vibration sensors. The collection of large signal datasets is enabling engineers to explore new and exciting deep learning applications.\r\n<h6><\/h6>\r\n<h3><span style=\"font-size: 16px;\"><strong>Signal Processing vs. Image Processing<\/strong><\/span><\/h3>\r\nDeep learning for\u00a0signal\u00a0data\u00a0typically\u00a0requires\u00a0preprocessing, transformation, and feature extraction steps that\u00a0image\u00a0processing\u00a0applications often do not. While large high-quality <em>image <\/em>datasets can be created with relatively low-cost cameras, signal sets are harder to obtain and will usually suffer from large variability caused by wideband noise, interference, non-linear trends, jitter, phase distortion, and missing samples.\r\n<h6><\/h6>\r\nWhile it is likely that you will be able to successfully train a deep model with a large set of high-quality raw images fed directly into a network, you will probably not succeed doing the same with a limited size, low SNR signal data set.\r\n<h6><\/h6>\r\n<h3><strong>Knowledge vs. Data<\/strong><\/h3>\r\nWe don\u2019t always have the luxury of having large signal data sets. Signals may be difficult to measure, or the application may not be easily observable. For example, consider the case of predicting remaining useful life of a machine. Fortunately, we live in a time where most machines work properly, so failure data will be less prevalent than healthy data.\r\n<h6><\/h6>\r\nBeing successful with deep learning for signal processing applications depends on your dataset size, your computational power, and how much knowledge you have about the data. You can visualize this in the figure below:\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2249 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/datavsknowledge1a-1024x661.png\" alt=\"\" width=\"1024\" height=\"661\" \/>\r\n<h6><\/h6>\r\nThe larger and higher quality the dataset, the closer you can get to being able to perform deep learning with raw signal data. Similarly, if you have higher computational power, you will be able to train more complex networks that will improve performance.\r\n<h6><\/h6>\r\nOn the other hand, if you understand your data, you may be successful extracting key features to feed into a traditional machine learning algorithm without the need of a large dataset and of a deep network. This requires more signal processing knowledge, but results can be impressive.\r\n<h6><\/h6>\r\nBetween the two extremes of raw signal data and crafting a customized machine learning approach are some interesting considerations:\r\n<ul>\r\n \t<li style=\"list-style: circle inside;\">A popular technique is to use time-frequency transformations to be able to use transfer learning on a pretrained convolutional neural network (CNN) similar to an image classification workflow. <em>(<a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/examples\/signal-classification-with-wavelet-analysis-and-convolutional-neural-networks.html\">See example<\/a>)<\/em><\/li>\r\n \t<li style=\"list-style: circle inside;\">Another is to preprocess the signal and extract some features to decrease the variability by increasing the prominence of the features that matter to the application. <em>(<a href=\"https:\/\/www.mathworks.com\/help\/signal\/examples\/classify-ecg-signals-using-long-short-term-memory-networks.html\">See example<\/a>)<\/em><\/li>\r\n \t<li style=\"list-style: circle inside;\">Another technique is to use wavelet scattering to automatically extract a low variant set of features that are likely to enhance the performance of your learning algorithm. <em>(<a href=\"\">See example<\/a>) <\/em><\/li>\r\n<\/ul>\r\nNote that adding feature extraction steps also offers other advantages, such as data-dimensionality reduction that can significantly reduce training and processing times.\r\n<h6><\/h6>\r\nAnother solution to having limited data sets is to augment them, or to generate synthetic signals using simulation, which we'll discuss more in the preprocessing section of the workflow.\r\n<h6><\/h6>\r\n<span style=\"font-size: 18px;\">Deep learning for signals workflow<\/span>\r\n<h6><\/h6>\r\nThe figure below depicts a typical end-to-end deep learning workflow for signal processing applications. Although most discussions about deep learning focus on the predictive model development, there are other equally important steps:\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2135 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/workflow.png\" alt=\"\" width=\"798\" height=\"384\" \/>\r\n<h6><\/h6>\r\nNext we\u2019ll walk through each of these steps, addressing what to do when:\r\n<h6><\/h6>\r\n<ol>\r\n \t<li>You don't have enough data, or the exact opposite problem: You have too much data that needs to be labeled.<\/li>\r\n \t<li>Feature extraction and signal transformations require domain knowledge. In cases where domain knowledge is limited, the availability of reference architectures helps to get a head start but such architectures are quite limited in signal processing applications.<\/li>\r\n \t<li>Deploying the trained model across multiple platforms is not an easy task, at it requires C or GPU code generation.<\/li>\r\n<\/ol>\r\nI'll also highlight new capabilities you can use in the latest release.\r\n<h6><\/h6>\r\n<span style=\"font-size: 16px;\"><strong>Manage Data<\/strong><\/span>\r\n<h6><\/h6>\r\nTypically, you will need to read your signal and label data, preprocess, transform, and extract features from it, then split it into training and testing sets, and feed it into a model.\r\n<h6><\/h6>\r\nHere are specific tools to manage your signal data sets:\r\n<h6><\/h6>\r\n<ul>\r\n \t<li><strong>Datastores<\/strong>: Datastores can help you read signal data from files without the risk of running out of memory. You can use the 'transform' method of a datastore to preprocess and extract features as you read the data. Datastores also split your dataset into training and tests sets. There are a few datastores to choose from depending on your file type: for example, audioDatastore, fileDatastore, and imageDatastore.<\/li>\r\n<\/ul>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td width=55%; style=\"vertical-align: middle;\"><strong>Signal Labeling:<\/strong> Supervised learning problems require data sets with ground truth labels. If your data set is large, adding labels can be a lengthy process. Signal Labeler app, <strong>(new in 19a)<\/strong>, helps to label signal attributes, regions, and points of interest. Then you can visualize and navigate through the labels in time.<\/td>\r\n<td><a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/signalLabelerFigure.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2187 size-large\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/signalLabelerFigure-1024x555.png\" alt=\"\" width=\"1024\" height=\"555\" \/><\/a><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<ul>\r\n \t<li><strong>Synthetic data generation and augmentation:<\/strong> As you start using your dataset to train a model, you may find that you need more data to improve accuracy or increase robustness. When recording and labeling real-world data is impractical or unreasonable, creating synthetic data sets through simulation is an option.\r\n<h6><\/h6>\r\nAnother common strategy consists of generating additional signals by applying variations, nuances, and transformations to the original dataset. For example, in a speech dataset consisting of spoken words, you can take one sample of a word, and alter its pitch\/frequency and duration, or you can add nuances such as background noise to effectively turn one data entry into several new entries.<\/li>\r\n<\/ul>\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2193 size-medium\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/ezgif.com-video-to-gif-8.gif\" alt=\"\" width=\"600\" height=\"400\" \/>\r\n<span style=\"font-size: 12px;\">Phased Array System Toolbox offers functionality to model micro-doppler radar signatures of pedestrians (new in 19a) that can be used in automated driving applications.<\/span>\r\n<h6><\/h6>\r\n<h3><span style=\"font-size: 16px;\"><strong>Visualize, Preprocess and Extract Features<\/strong><\/span><\/h3>\r\nWhen you are trying to figure out what type of preprocessing and feature extraction you need for your signals, a common first step is to visualize and explore the signals in multiple domains.\r\n<h6><\/h6>\r\nSignal Analyzer app allows you to navigate through signals in the time, frequency, and time-frequency domains, extract regions of interest, transform the data and explore preprocessing steps.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"wp-image-2087 size-full alignleft\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/analyzer_gif.gif\" alt=\"\" width=\"600\" height=\"374\" \/>\r\n<h6><\/h6>\r\nThere are other apps as well, including:\r\n<ul>\r\n \t<li>Signal Multiresolution Analyzer, which decomposes signals into time-aligned components. This allows you to pick only the signal components that matter and use these to train a model.<\/li>\r\n \t<li>Signal Denoiser, (included in Wavelet Toolbox) which helps to smooth and denoise signals.<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\nAll these apps generate MATLAB code, which you can then use to process your entire dataset after interactively determining the right approach.\r\n<h6><\/h6>\r\n<h3><span style=\"font-size: 16px;\"><strong>Transform &amp; Extract Features<\/strong><\/span><\/h3>\r\n<h6><\/h6>\r\nWhile apps make getting started easier, users familiar with Signal Processing Toolbox and Wavelet Toolbox will know we also have hundreds of command line functions to enable more advanced workflows. We have time-domain feature extraction tools to find signal patterns, detect signal envelopes and change points.\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2297 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/features111-2.png\" alt=\"\" width=\"500\" height=\"387\" \/>\r\n<h6><\/h6>\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\nThere are a variety of spectral functions to measure power, bandwidth and harmonics.\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n<img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2301 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/features2222-1.png\" alt=\"\" width=\"500\" height=\"387\" \/>\r\n<h6><\/h6>\r\nAnd there are time-frequency transformation functions, from continuous wavelet transform, to Wigner-Ville distribution and short-time Fourier transform, which are commonly used in deep learning applications.\r\n\r\nYou can choose the transform that best suits your time and frequency resolution needs.\r\n<p style=\"text-align: center; font-size: 16px;\"><\/p>\r\n\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter wp-image-2303 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/features333.png\" alt=\"\" width=\"500\" height=\"387\" \/><\/h6>\r\nThere are also functions to extract features specific to audio and speech, radar, sensor fusion and text analytics applications, which we will save for a future post.\r\n\r\nLinks to signal-specific processing functionality can be found here:\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/signal\/index.html\">Signal Processing Toolbox<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/index.html\">Wavelet Toolbox<\/a><\/li>\r\n<\/ul>\r\n<strong>Wavelet Scattering:<\/strong>\r\n\r\nA wavelet scattering network (new in 18a) is a deep network with fixed weights that are set to wavelets.\u00a0 This feature extractor returns low variant features that can be used in conjunction with a machine or deep learning model to produce highly accurate results with few parameters to tweak.\r\n\r\nWavelet scattering is a great alternative for non-signal processing experts as it automatically extracts relevant features without the need to have expertise on the nature of the data.\r\n\r\nTo see wavelet scattering in action, here are some examples:\r\n<ul>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/examples\/ecg-signal-classification-using-wavelet-time-scattering.html\">Wavelet Time Scattering for ECG Signal Classification<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/examples\/music-genre-classification-using-wavelet-scattering.html\">Music Genre Classification Using Wavelet Time Scattering<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/examples\/wavelet-time-scattering-classification-of-phonocardiogram-data.html\">Wavelet Time Scattering Classification of Phonocardiogram Data<\/a><\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/examples\/spoken-digit-recognition-with-wavelet-scattering-and-deep-learning.html\">Spoken Digit Recognition with Wavelet Scattering and Deep Learning<\/a><\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<h3><span style=\"font-size: 16px;\"><strong>Code Generation<\/strong><\/span><\/h3>\r\n<h6><\/h6>\r\nIn 19a, there are significant advancements to automatic C code generation support for signal processing workflows, including support for filtering functions, spectral analysis, and the continuous wavelet transform (cwt). In 19a Wavelet Toolbox also added GPU code generation support for cwt. A complete list of signal processing and wavelet supported functions is here (<a href=\"https:\/\/www.mathworks.com\/help\/signal\/referencelist.html?type=function&amp;s_cid=doc_ftr&amp;capability=codegen\">Signal Processing Toolbox code generation support<\/a>, <a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/referencelist.html?type=function&amp;s_cid=doc_ftr&amp;capability=codegen\">Wavelet Toolbox code generation support<\/a>).\r\n\r\n<strong>List of new examples<\/strong>\r\n\r\nFinally, to see a list of new examples for deep learning for signal processing applications you can visit the following pages:\r\n<ul>\r\n \t<li>Signal Processing (<a href=\"https:\/\/www.mathworks.com\/help\/signal\/machine-learning-and-deep-learning.html\">link<\/a>)<\/li>\r\n \t<li>Wavelet (<a href=\"https:\/\/www.mathworks.com\/help\/wavelet\/machine-learning-and-deep-learning.html\">link<\/a>)<\/li>\r\n \t<li>Communications (<a href=\"https:\/\/www.mathworks.com\/help\/comm\/examples\/modulation-classification-with-deep-learning.html?searchHighlight=modulation%20class&amp;s_tid=doc_srchtitle\">link<\/a>)<\/li>\r\n \t<li>Audio (<a href=\"https:\/\/www.mathworks.com\/help\/audio\/measurements-and-feature-extraction.html\">link<\/a>)<\/li>\r\n<\/ul>\r\nHopefully you now have a better sense of the signal processing workflow for deep learning, and the tools we offer to enable it and make it easier.\r\n\r\nHave any questions about signal processing for deep learning? Leave a comment below.\r\n<h6><\/h6>\r\n<h6><\/h6>\r\n<p><a href=\"https:\/\/twitter.com\/jo_pings?ref_src=twsrc%5Etfw\" class=\"twitter-follow-button\" data-size=\"large\" data-show-count=\"false\">Follow @jo_pings<\/a><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/05\/deep-learning-for-signal-white-paper-banner_smaller.jpg\" onError=\"this.style.display ='none';\" \/><\/div><p>I\u2019d like to introduce Frantz Bouchereau, development manager for Signal Processing Toolbox who is going to dive deep into insights on deep learning for signal processing, including the complete deep... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2019\/05\/13\/deep-learning-for-signal-processing-applications\/\">read more >><\/a><\/p>","protected":false},"author":156,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/2081"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/156"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=2081"}],"version-history":[{"count":84,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/2081\/revisions"}],"predecessor-version":[{"id":2317,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/2081\/revisions\/2317"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=2081"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=2081"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=2081"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}