The R2017b release of MathWorks products shipped just two weeks ago, and it includes many new capabilities for deep learning. Developers on several product teams have been working hard on these capabilities, and everybody is excited to see them make it into your hands. Today, I'll give you a little tour of what you can expect when you get a chance to update to the new release.
The heart of deep learning for MATLAB is, of course, the Neural Network Toolbox. The Neural Network Toolbox introduced two new types of networks that you can build and train and apply: directed acyclic graph (DAG) networks, and long short-term memory (LSTM) networks.
In a DAG network, a layer can have inputs from multiple layers instead of just one one. A layer can also output to multiple layers. Here's a sample from the example Create and Train DAG Network for Deep Learning.
You can try out the pretrained GoogLeNet model, which is a DAG network that you can load using googlenet.
Experiment also with long short-term memory (LSTM) networks, which have the ability to learn long-term dependencies in time-series data.
There's a pile of new layer types, too: batch normalization, transposed convolution, max unpooling, leaky ReLU, clipped rectified ReLU, addition, and depth concatenation.
My colleague Joe used the Neural Network Toolbox to define his own type of network layer based on a paper he read a couple of months ago. I'll show you his work in detail a little later this fall.
When you train your networks, you can now plot the training progress. You can also validate network performance and automatically halt training based on the validation metrics. Plus, you can find optimal network parameters and training options using Bayesian optimization.
Automatic image preprocessing and augmentation is now available for network training. Image augmentation is the idea of increasing the training set by randomly applying transformations, such as resizing, rotation, reflection, and translation, to the available images.
As an image processing algorithms person, I am especially intrigued by the new semantic segmentation capability, which lets you classify pixel regions and visualize the results.
If you are implementing deep learning methods in embedded system, take a look at GPU Coder, a brand new product in the R2017b release. GPU Coder generates CUDA from MATLAB code for deep learning, embedded vision, and autonomous systems. The generated code is well optimized, as you can see from this performance benchmark plot.
MathWorks benchmarks of inference performance of AlexNet using GPU acceleration, Titan XP GPU, Intel® Xeon® CPU E5-1650 v4 at 3.60GHz, cuDNN v5, and Windows 10. Software versions: MATLAB (R2017b), TensorFlow (1.2.0), MXNet (0.10), and Caffe2 (0.8.1).
I have just scratched the surface of the deep learning capabilities in the ambitious R2017b release.
Here are some additional sources of information.
Get the MATLAB code
Published with MATLAB® R2017b
5 CommentsOldest to Newest
Can we use the deep parts of the NN toolkit for function approximation? I would like to take a data set which has 26 inputs and 1 output and use a deep NN to approximate it.
I tried to install MATLAB 2018 trial package for deep learning but could not do so. I need help in this regard.
I would like to ask is it possible to develop deep learning code in 2017b package?
Shubhangi—Trial information can be obtained here; I don’t know more about it than that. Yes, you can do deep learning with the R2017b release.
These benchmarks are highly suspicious. Tensorflow is written in C++ and utilizes the Nvidia CUDA libraries which are highly optimized for neural networks. I believe these tests were skewed. The Tensorflow implementation was possibly coded inefficiently or this is a special case where MATLAB outperforms Tensorflow.
Will—Neural Network Toolbox and Parallel Computing Toolbox functions are also based on C++ implementations and NVIDIA CUDA libraries for optimized GPU execution.