Convert Deep Learning Models between PyTorch, TensorFlow, and MATLAB
- Import models from TensorFlow and PyTorch into MATLAB
- Export models from MATLAB to TensorFlow and PyTorch
What? | Import PyTorch models. | Import TensorFlow models. | Import PyTorch and Tensorflow models interactively. |
How? | Use the importNetworkFromPyTorch function. | Use the importNetworkFromTensorFlow function. | Use the Deep Network Designer app. |
When? | Introduced in R2022b. | Introduced in R2023b. | Import capability introduced in R2023b. |
Quick Example
This example shows you how to import an image classification model from PyTorch. The PyTorch model must be pretrained and traced. Run the following code in Python to get and save the PyTorch model. Load the MnasNet pretrained image classification model from the TorchVision library.import torch from torchvision import models model = models.mnasnet1_0(pretrained=True)Trace the PyTorch model. For more information on how to trace a PyTorch model, go to Torch documentation: Tracing a function. Then, save the PyTorch model.
X = torch.rand(1,3,224,224) traced_model = torch.jit.trace(model.forward,X) traced_model.save("traced_mnasnet1_0.pt")Now, go to MATLAB and import the model by using the importNetworkFromPyTorch function. Specify the name-value argument PyTorchInputSizes so that the import function automatically creates and adds the input layer for a batch of images.
net = importNetworkFromPyTorch("mnasnet1_0.pt",PyTorchInputSizes=[NaN,3,224,224])
net = dlnetwork with properties: Layers: [153×1 nnet.cnn.layer.Layer] Connections: [162×2 table] Learnables: [210×3 table] State: [104×3 table] InputNames: {'InputLayer1'} OutputNames: {'aten__linear12'} Initialized: 1 View summary with summary.Read the image you want to classify. Resize the image to the input size of the network.
Im_og = imread("peacock.jpg"); InputSize = [224 224 3]; Im = imresize(Im_og,InputSize(1:2));The inputs to MnasNet require further preprocessing. Rescale the image. Then, normalize the image by subtracting the training images mean and dividing by the training images standard deviation. For more information, see Input Data Preprocessing.
Im = rescale(Im,0,1); meanIm = [0.485 0.456 0.406]; stdIm = [0.229 0.224 0.225]; Im = (Im - reshape(meanIm,[1 1 3]))./reshape(stdIm,[1 1 3]);Convert the image to a dlarray object. Format the image with the dimensions "SSCB" (spatial, spatial, channel, batch).
Im_dlarray = dlarray(single(Im),"SSCB");Classify the image and find the predicted label.
prob = predict(net,Im_dlarray); [~,label_ind] = max(prob);Display the image and classification result.
imshow(Im_og) title(strcat("Classification Result: ",ClassNames(label_ind)),FontSize=16)Importing a model from TensorFlow is quite similar to importing a model from PyTorch. Of course you need to use the importNetworkFromTensorFlow function instead. Note that your TensorFlow model must be in the SavedModel format and saved by using the following code in Python.
model.save("myModelTF")For more examples, check out the reference pages of the importNetworkFromPyTorch and importNetworkFromTensorFlow functions, and the documentation page on Pretrained Networks from External Platforms.
Why Import Models into MATLAB?
In short, import models into MATLAB because an imported network is a MATLAB network. This means, you can perform all the following tasks with built-in tools. To learn more, see Deep Learning Toolbox.Task | Example | More Resources | |
Easily perform transfer learning and retrain the network. | Prepare Network for Transfer Learning Using Deep Network Designer | Import Deep Neural Network for Transfer Learning | |
Visualize activations, explain decisions, and verify the robustness of the network (with Deep Learning Toolbox Verification Library). | Understand Network Predictions Using LIME | Visualize and Verify Deep Neural Networks | |
Simulate the network in Simulink and test its performance within a larger system. | Classify Images in Simulink with Imported TensorFlow Network | Deep Learning with Simulink | |
Compress the network with quantization, projection, or pruning. | Compress Neural Network Using Projection | Quantization, Projection, and Pruning | |
Automatically generate C/C++, CUDA, and HDL code for the network. | Code Generation for Deep Learning Networks | Code Generation and Deployment |
What? | Export models to TensorFlow. | Export models to PyTorch. |
How? | Use the exportNetworkToTensorFlow function. | Export via ONNX by using the exportONNXNetwork function. |
When? | Introduced in R2022b. | ONNX export introduced in R2018a. |
Quick Example
Load the pretrained SqueezeNet convolutional network by using imagePretrainedNetwork. This function, which was introduced in R2024a, is the easiest (and recommended) way to load pretrained image classification networks.[net,ClassNames] = imagePretrainedNetwork("squeezenet");Export the network net to TensorFlow. The exportNetworkToTensorFlow function saves the TensorFlow model in the Python package myModel.
exportNetworkToTensorFlow(net,"myModel")And you are done with exporting! If you want to test model in Python, you can use an image available in MATLAB. Read the image you want to classify. Resize the image to the input size of the network.
Im = imread("peacock.jpg"); InputSize = net.Layers(1).InputSize; Im = imresize(Im,InputSize(1:2));Permute the 2-D image data from the MATLAB ordering (HWCN) to the TensorFlow ordering (NHWC), where H, W, and C are the height, width, and number of channels of the image, respectively, and N is the number of images. For more information, see Input Dimension Ordering. Save the image in a MAT file.
ImTF = permute(Im,[4,1,2,3]); filename = "peacock.mat"; save(filename,"ImTF")The code below shows you how to use the exported model to predict in Python or save it in the SavedModel format. First, load the exported TensorFlow model from the package myModel.
import myModel model = myModel.load_model()Save the exported model in the TensorFlow SavedModel format. Saving the model in SavedModel format is optional. You can perform deep learning workflows directly with the model.
model.save("myModelTF")Classify the image with the exported model.
import scipy.io as sio x = sio.loadmat("peacock.mat") x = x["ImTF"] import numpy as np x_np = np.asarray(x, dtype=np.float32) scores = model.predict(x_np)Read more on exporting deep neural networks to TensorFlow in this blog post. For more examples, see the exportNetworkToTensorFlow and exportONNXNetwork reference pages and the documentation page on Exporting Deep Neural Networks.
Comments
To leave a comment, please click here to sign in to your MathWorks Account or create a new one.