{"id":7007,"date":"2021-04-16T09:14:49","date_gmt":"2021-04-16T13:14:49","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=7007"},"modified":"2022-04-11T13:25:30","modified_gmt":"2022-04-11T17:25:30","slug":"bringing-tensorflow-models-into-matlab","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2021\/04\/16\/bringing-tensorflow-models-into-matlab\/","title":{"rendered":"Bringing TensorFlow Models into MATLAB"},"content":{"rendered":"The following is a post from Shounak Mitra, Product Manager for Deep Learning Toolbox, here to talk about practical ways to work with TensorFlow and MATLAB.<\/em>\r\n
<\/h6>\r\n

In release R2021a, a converter for TensorFlow models<\/a> was released as a support package supporting import of TensorFlow 2 models into Deep Learning Toolbox. In this blog, we will explore the ways you can use the converter for TensorFlow models and do the following:<\/p>\r\n\r\n

    \r\n \t
  1. Visualize and analyze the network<\/span><\/a><\/li>\r\n \t
  2. Generate C\/C++\/CUDA code<\/span><\/a><\/li>\r\n \t
  3. Integrate the network with Simulink<\/span><\/a><\/li>\r\n<\/ol>\r\n

    To bring models trained in TensorFlow 2 into MATLAB, you can use the function importTensorFlowNetwork<\/span><\/a>, which enables you to import the model and its weights into MATLAB. (Note: you can also use importTensorFlowLayers<\/span><\/a> to import layers from TensorFlow).<\/p>\r\n\r\n\r\n\r\n\r\n\r\n

    <\/h6> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>
    <\/h6>To see a more-detailed post on how <\/strong>to bring in TensorFlow model into MATLAB, check out this related post<\/u><\/a> on bringing in TensorFlow (and other) networks
    <\/h6> >>>>>>>>>>>>>>>>>>>>>>>>>>>>><\/p><\/td>\r\n
    		Related Post: Importing Models from TensorFlow, PyTorch, and ONNX<\/a> \"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n\r\nThe rest of this blog post will focus on what you can do with TensorFlow Models after <\/em>they are brought into MATLAB.<\/p>\r\n\r\n
    <\/h6>\r\n

    Figure 1: Common workflows after importing TensorFlow model into MATLAB<\/strong><\/p>\r\n\r\n

    <\/h6>\r\n1. Visualize and analyze the network<\/a><\/span>\r\n
    <\/h6>\r\n

    To understand the network, we'll use Deep Network Designer app to visualize the network architecture. To load up the app, type deepNetworkDesigner<\/span> in the command line and load the network from workspace. Once imported into the app, the network looks like Figure 4a. The layer architecture contains skip-connections which is typical of ResNet architectures. You can at this stage use this network for transfer learning workflows. Check this video out to learn how to interactively modify a deep learning network for transfer learning<\/a>. You can also click on the Analyze<\/em> button in the app (Figure 4b) and investigate the activation sizes and see if the network has errors like incorrect tensor shapes, misplaced connections, etc.<\/p>\r\n

    \"\"<\/p>\r\n

    Figure 4a:\u00a0ResNet50 architecture inside the Deep Network Designer app<\/strong><\/p>\r\n

    \"\"<\/a><\/p>\r\n\r\n

    <\/h6>\r\n

    Figure 4b: Analyze the imported network for errors and visualize the key components in the architecture \u2013 the skipped connections in the case of resnet50. Click for larger view.\u00a0<\/strong><\/p>\r\n\r\n

    <\/h6>\r\n2. Generate C\/C++\/CUDA code<\/a><\/span>\r\n
    <\/h6>\r\n

    One of the most common paths our customers take after importing a model is generating code, targeting different hardware platforms.<\/p>\r\n

    In this example we'll generate CUDA code, using GPU Coder<\/a>, targeting the cuDNN library in 3 easy steps.<\/p>\r\n\r\n

    Step 1: Verify the GPU environment<\/h2>\r\n

    This performs a complete check of all third-party tools required for GPU code generation. The output shown here is representative. Your results might differ.<\/p>\r\n\r\n\r\n\r\n\r\n
    \r\n
    envCfg = coder.gpuEnvConfig('host');\r\nenvCfg.DeepLibTarget = 'cudnn';\r\nenvCfg.DeepCodegen = 1;\r\nenvCfg.Quiet = 1;\r\ncoder.checkGpuInstall(envCfg);\r\n<\/pre>\r\n<\/td>\r\n
    		\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
    <\/h6>\r\n

    Figure 5:<\/strong> Verify the GPU environment to make sure all the essential libraries are available<\/strong><\/p>\r\n\r\n

    Step 2: Define Entry Point Function<\/h2>\r\n

    The\u00a0resnet50_predict.m<\/span>\u00a0entry-point function takes an image input and runs prediction on the image using the imported ResNet50 model. The function uses a persistent object\u00a0mynet<\/em>\u00a0to load the series network object and reuses the persistent object for prediction on subsequent calls.<\/p>\r\n\r\n

    function out = resnet50_predict(in)\r\n%#codegen <\/span>\r\npersistent mynet;\r\n\r\nif isempty(mynet)\r\n  mynet = coder.loadDeepLearningNetwork('resnet50.mat','net');\r\nend\r\n\r\n% pass in input <\/span>\r\nout = mynet.predict(in);\r\n\r\n<\/pre>\r\n
    <\/h6>\r\n
    Step 3: Run MEX Code Generation<\/h2>\r\n
    Call the entry point function and generate C++ code targeting cudnn libraries<\/p>\r\n
    To generate CUDA code for the\u00a0resnet50_predict.m<\/span> entry-point function, create a GPU code configuration object for a MEX target and set the target language to C++. Use the\u00a0coder.DeepLearningConfig<\/a>\u00a0function to create a\u00a0CuDNN\u00a0deep learning configuration object and assign it to the\u00a0DeepLearningConfig\u00a0property of the GPU code configuration object. Run the\u00a0codegen\u00a0command and specify an input size of [224,224,3]. This value corresponds to the input layer size of the ResNet50 network.<\/p>\r\nRun MEX\r\n
    cfg = coder.gpuConfig('mex');\r\ncfg.TargetLang = 'C++';\r\ncfg.DeepLearningConfig = coder.DeepLearningConfig('cudnn');\r\ncodegen -config cfg resnet50_predict -args {ones(224,224,3)} -report\r\n<\/pre>\r\n
    GPU Coder creates a code generation report that provides an interface to examine the original MATLAB code and generated CUDA code. The report also provides a handy interactive code traceability tool to map between MATLAB code and CUDA. Figure 6 shows a screen capture of the tool in action.<\/p>\r\n
    \"\"<\/a><\/p>\r\n
    Figure 6:<\/strong> Generated report for code generation. Click to see details.<\/strong><\/p>\r\n
    In this example, we targeted the cuDNN libraries. You can also target Intel and ARM CPUs using MATLAB Coder<\/a> and FPGAs and SoCs using Deep Learning HDL Toolbox<\/a>.<\/p>\r\n\r\n
    <\/h6>\r\n3. Integrate the Network with Simulink<\/a><\/span>\r\n
    <\/h6>\r\n
    Often, deep learning models are used as a component in bigger systems. Simulink helps explore a wide design space by modeling the system under test and the physical plant where you can use one multi-domain environment to simulate how all parts of the system behave. In this section, we will see how the resnet50 model imported from TensorFlow can be integrated into Simulink.<\/p>\r\n
    We'll integrate this model with Simulink in 3 easy steps. But first, save the Resnet50 model in your directory in MATLAB. Use save('resnet50.mat','net')<\/span> to do so.<\/p>\r\n\r\n
    Step 1: Open Simulink and Access Library Browser<\/h2>\r\n