{"id":14676,"date":"2024-04-22T09:59:48","date_gmt":"2024-04-22T13:59:48","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=14676"},"modified":"2024-05-29T15:46:08","modified_gmt":"2024-05-29T19:46:08","slug":"convert-deep-learning-models-between-pytorch-tensorflow-and-matlab","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2024\/04\/22\/convert-deep-learning-models-between-pytorch-tensorflow-and-matlab\/","title":{"rendered":"Convert Deep Learning Models between PyTorch, TensorFlow, and MATLAB"},"content":{"rendered":"
<\/h6>\r\nIn this blog post we are going to show you how to use the newest MATLAB functions to:\r\n
<\/h6>\r\n
    \r\n \t
  1. Import models from TensorFlow and PyTorch into MATLAB<\/a><\/li>\r\n \t
  2. Export models from MATLAB to TensorFlow and PyTorch<\/a><\/li>\r\n<\/ol>\r\nThis is a brief blog post that points you to the right functions and other resources for converting deep learning models between MATLAB, PyTorch\u00ae, and TensorFlow\u2122. Two good resources to get started with are the documentation topics Interoperability Between Deep Learning Toolbox, TensorFlow, PyTorch, and ONNX<\/a> and Tips on Importing Models from TensorFlow, PyTorch, and ONNX<\/a>.\r\n
    <\/h6>\r\nIf you have any questions about the functionality presented in this blog post or want to share the exciting projects for which you are using model conversion, comment below.\r\n
    <\/h6>\r\n \r\n
    <\/h6>\r\n

    <\/a>Import Models into MATLAB<\/strong><\/p>\r\nYou can import models from PyTorch or TensorFlow with just one line of code.\r\n

    <\/h6>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    <\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n
    What?<\/strong><\/td>\r\nImport PyTorch models.<\/td>\r\nImport TensorFlow models.<\/td>\r\nImport PyTorch and Tensorflow models interactively.<\/td>\r\n<\/tr>\r\n
    How?<\/strong><\/td>\r\nUse the importNetworkFromPyTorch<\/a> function.<\/td>\r\nUse the importNetworkFromTensorFlow<\/a> function.<\/td>\r\nUse the Deep Network Designer app<\/a>.<\/td>\r\n<\/tr>\r\n
    When?<\/strong><\/td>\r\nIntroduced in R2022b.<\/td>\r\nIntroduced in R2023b.<\/td>\r\nImport capability introduced in R2023b.<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\n

    Quick Example<\/strong><\/p>\r\nThis example shows you how to import an image classification model from PyTorch. The PyTorch model must be pretrained and traced.\r\n

    <\/h6>\r\nRun the following code in Python to get and save the PyTorch model. Load the MnasNet pretrained image classification model from the\u00a0TorchVision library<\/a>.\r\n
    <\/h6>\r\n
    import torch\r\nfrom torchvision import models\r\nmodel = models.mnasnet1_0(pretrained=True)\r\n<\/pre>\r\n
    <\/h6>\r\nTrace the PyTorch model. For more information on how to trace a PyTorch model, go to Torch documentation: Tracing a function<\/a>. Then, save the PyTorch model.\r\n
    <\/h6>\r\n
    X = torch.rand(1,3,224,224)\r\ntraced_model = torch.jit.trace(model.forward,X)\r\ntraced_model.save(\"traced_mnasnet1_0.pt\")\r\n<\/pre>\r\n
    <\/h6>\r\nNow, go to MATLAB and import the model by using the importNetworkFromPyTorch function. Specify the name-value argument PyTorchInputSizes so that the import function automatically creates and adds the input layer for a batch of images.\r\n
    <\/h6>\r\n
    net = importNetworkFromPyTorch(\"mnasnet1_0.pt\",PyTorchInputSizes=[NaN,3,224,224])\r\n<\/pre>\r\n
    <\/h6>\r\n
    net = \r\n\r\n  dlnetwork<\/a> with properties:\r\n\r\n         Layers: [153\u00d71 nnet.cnn.layer.Layer]\r\n    Connections: [162\u00d72 table]\r\n     Learnables: [210\u00d73 table]\r\n          State: [104\u00d73 table]\r\n     InputNames: {'InputLayer1'}\r\n    OutputNames: {'aten__linear12'}\r\n    Initialized: 1\r\n\r\n  View summary with summary<\/a>.\r\n\r\n<\/pre>\r\n
    <\/h6>\r\nRead the image you want to classify. Resize the image to the input size of the network.\r\n
    <\/h6>\r\n
    Im_og = imread(\"peacock.jpg\");\r\nInputSize = [224 224 3];\r\nIm = imresize(Im_og,InputSize(1:2));\r\n<\/pre>\r\n
    <\/h6>\r\nThe inputs to MnasNet require further preprocessing. Rescale the image. Then, normalize the image by subtracting the training images mean and dividing by the training images standard deviation. For more information, see\u00a0Input Data Preprocessing<\/a>.\r\n
    <\/h6>\r\n
    Im = rescale(Im,0,1);\r\n\r\nmeanIm = [0.485 0.456 0.406];\r\nstdIm = [0.229 0.224 0.225];\r\nIm = (Im - reshape(meanIm,[1 1 3])).\/reshape(stdIm,[1 1 3]);\r\n<\/pre>\r\n
    <\/h6>\r\nConvert the image to a\u00a0dlarray<\/a>\u00a0object. Format the image with the dimensions\u00a0\"SSCB\"\u00a0(spatial, spatial, channel, batch).\r\n
    Im_dlarray = dlarray(single(Im),\"SSCB\");\r\n<\/pre>\r\n
    <\/h6>\r\nClassify the image and find the predicted label.\r\n
    <\/h6>\r\n
    prob = predict(net,Im_dlarray);\r\n[~,label_ind] = max(prob);\r\n<\/pre>\r\n
    <\/h6>\r\nDisplay the image and classification result.\r\n
    <\/h6>\r\n
    imshow(Im_og)\r\ntitle(strcat(\"Classification Result: \",ClassNames(label_ind)),FontSize=16)\r\n<\/pre>\r\n
    <\/h6>\r\n\"\"\r\n
    <\/h6>\r\n \r\n
    <\/h6>\r\nImporting a model from TensorFlow is quite similar to importing a model from PyTorch. Of course you need to use the importNetworkFromTensorFlow function instead. Note that your TensorFlow model must be in the SavedModel format and saved by using the following code in Python.\r\n
    <\/h6>\r\n
    model.save(\"myModelTF\")\r\n<\/pre>\r\n
    <\/h6>\r\nFor more examples, check out the reference pages of the importNetworkFromPyTorch<\/a> and importNetworkFromTensorFlow<\/a> functions, and the documentation page on Pretrained Networks from External Platforms<\/a>.\r\n
    <\/h6>\r\n \r\n
    <\/h6>\r\n
    Why Import Models into MATLAB?<\/strong><\/p>\r\nIn short, import models into MATLAB because an imported network is a MATLAB network. This means, you can perform all the following tasks with built-in tools. To learn more, see Deep Learning Toolbox<\/a>.\r\n
    <\/h6>\r\n \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    <\/td>\r\nTask<\/strong><\/td>\r\nExample<\/strong><\/td>\r\nMore Resources<\/strong><\/td>\r\n<\/tr>\r\n
    \"\"<\/td>\r\nEasily perform transfer learning<\/a> and retrain the network.<\/td>\r\nPrepare Network for Transfer Learning Using Deep Network Designer<\/a><\/td>\r\nImport Deep Neural Network for Transfer Learning<\/a><\/td>\r\n<\/tr>\r\n
    \"\"<\/td>\r\nVisualize activations, explain decisions, and verify the robustness of the network (with Deep Learning Toolbox Verification Library<\/a>).<\/td>\r\nUnderstand Network Predictions Using LIME<\/a><\/td>\r\nVisualize and Verify Deep Neural Networks<\/a><\/td>\r\n<\/tr>\r\n
    \"\"<\/td>\r\nSimulate the network in Simulink and test its performance within a larger system.<\/td>\r\nClassify Images in Simulink with Imported TensorFlow Network<\/a><\/td>\r\nDeep Learning with Simulink<\/a><\/td>\r\n<\/tr>\r\n
    \"\"<\/td>\r\nCompress the network with quantization, projection, or pruning.<\/td>\r\nCompress Neural Network Using Projection<\/a><\/td>\r\nQuantization, Projection, and Pruning<\/a><\/td>\r\n<\/tr>\r\n
    \"\"<\/td>\r\nAutomatically generate C\/C++, CUDA, and HDL code for the network.<\/td>\r\nCode Generation for Deep Learning Networks<\/a><\/td>\r\nCode Generation and Deployment<\/a><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\n \r\n
    <\/h6>\r\n
    <\/a>Export Models from MATLAB<\/strong><\/p>\r\nYou can export models to TensorFlow directly. To export a model to PyTorch, you must first convert the model to the ONNX model format.\r\n
    <\/h6>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
    <\/td>\r\n\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n
    What?<\/strong><\/td>\r\nExport models to TensorFlow.<\/td>\r\nExport models to PyTorch.<\/td>\r\n<\/tr>\r\n
    How?<\/strong><\/td>\r\nUse the exportNetworkToTensorFlow<\/a> function.<\/td>\r\nExport via ONNX by using the exportONNXNetwork<\/a> function.<\/td>\r\n<\/tr>\r\n
    When?<\/strong><\/td>\r\nIntroduced in R2022b.<\/td>\r\nONNX export introduced in R2018a.<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\n \r\n
    <\/h6>\r\n
    Quick Example<\/strong><\/p>\r\nLoad the pretrained\u00a0SqueezeNet\u00a0convolutional network by using imagePretrainedNetwork<\/a>. This function, which was introduced in R2024a, is the easiest (and recommended) way to load pretrained image classification networks.\r\n
    <\/h6>\r\n
    [net,ClassNames] = imagePretrainedNetwork(\"squeezenet\");\r\n<\/pre>\r\n
    <\/h6>\r\nExport the network net\u00a0to TensorFlow. The\u00a0exportNetworkToTensorFlow\u00a0function saves the TensorFlow model in the Python package\u00a0myModel.\r\n
    <\/h6>\r\n
    exportNetworkToTensorFlow(net,\"myModel\")\r\n<\/pre>\r\nAnd you are done with exporting!\r\n
    <\/h6>\r\nIf you want to test model in Python, you can use an image available in MATLAB. Read the image you want to classify. Resize the image to the input size of the network.\r\n
    <\/h6>\r\n
    Im = imread(\"peacock.jpg\");\r\nInputSize = net.Layers(1).InputSize;\r\nIm = imresize(Im,InputSize(1:2));\r\n<\/pre>\r\n
    <\/h6>\r\nPermute the 2-D image data from the MATLAB ordering (HWCN) to the TensorFlow ordering (NHWC), where\u00a0H,\u00a0W, and\u00a0C\u00a0are the height, width, and number of channels of the image, respectively, and\u00a0N\u00a0is the number of images. For more information, see Input Dimension Ordering<\/a>. Save the image in a MAT file.\r\n
    <\/h6>\r\n
    ImTF = permute(Im,[4,1,2,3]);\r\nfilename = \"peacock.mat\";\r\nsave(filename,\"ImTF\")\r\n<\/pre>\r\n
    <\/h6>\r\nThe code below shows you how to use the exported model to predict in Python or save it in the SavedModel format<\/a>. First, load the exported TensorFlow model from the package myModel.\r\n
    <\/h6>\r\n
    import myModel\r\nmodel = myModel.load_model()\r\n<\/pre>\r\n
    <\/h6>\r\nSave the exported model in the TensorFlow SavedModel format. Saving the model in SavedModel format is optional. You can perform deep learning workflows directly with the model.\r\n
    <\/h6>\r\n
    model.save(\"myModelTF\")\r\n<\/pre>\r\n
    <\/h6>\r\nClassify the image with the exported model.\r\n
    <\/h6>\r\n
    import scipy.io as sio\r\nx = sio.loadmat(\"peacock.mat\")\r\nx = x[\"ImTF\"]\r\n\r\nimport numpy as np\r\nx_np = np.asarray(x, dtype=np.float32)\r\n\r\nscores = model.predict(x_np)\r\n<\/pre>\r\n
    <\/h6>\r\nRead more on exporting deep neural networks to TensorFlow in this blog post<\/a>. For more examples, see the exportNetworkToTensorFlow<\/a> and exportONNXNetwork<\/a> reference pages and the documentation page on Exporting Deep Neural Networks<\/a>.\r\n
    <\/h6>\r\n \r\n
    <\/h6>","protected":false},"excerpt":{"rendered":"
    \"\"<\/div>
    \r\nIn this blog post we are going to show you how to use the newest MATLAB functions to:\r\n\r\n\r\n \tImport models from TensorFlow and PyTorch into MATLAB\r\n \tExport models from MATLAB to TensorFlow and... read more >><\/a><\/p>","protected":false},"author":194,"featured_media":15386,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9,5,39,59,45,42],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/14676"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/194"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=14676"}],"version-history":[{"count":57,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/14676\/revisions"}],"predecessor-version":[{"id":15389,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/14676\/revisions\/15389"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media\/15386"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=14676"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=14676"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=14676"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}