{"id":12857,"date":"2023-07-26T09:40:37","date_gmt":"2023-07-26T13:40:37","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=12857"},"modified":"2024-08-21T08:22:52","modified_gmt":"2024-08-21T12:22:52","slug":"matlab-with-tensorflow-from-your-jupyter-notebook","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2023\/07\/26\/matlab-with-tensorflow-from-your-jupyter-notebook\/","title":{"rendered":"MATLAB with TensorFlow from Your Jupyter Notebook"},"content":{"rendered":"<h6><\/h6>\r\nThe MATLAB Kernel for Jupyter\u00ae (<a href=\"https:\/\/github.com\/mathworks\/jupyter-matlab-proxy\">GitHub: jupyter-matlab-proxy<\/a>) was released a few months ago. The MATLAB Kernel for Jupyter now <a href=\"https:\/\/blogs.mathworks.com\/matlab\/2023\/06\/26\/matlab-kernel-for-jupyter-now-with-windows-support\/\">supports Windows\u00ae<\/a>, in addition to macOS\u00ae and Linux\u00ae. In this blog post, <a href=\"https:\/\/www.linkedin.com\/in\/yanndebray\/\">Yann Debray<\/a> and I will show how you can create a deep learning model and convert it from MATLAB to TensorFlow\u2122 <strong>by running MATLAB code<\/strong> and train the converted TensorFlow model <strong>by running Python code<\/strong>; all from the <strong>same Jupyter notebook<\/strong>.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-15395 \" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/pt_to_ml_inJupyter.png\" alt=\"Import a Pytorch model into MATLAB using Jupyter Notebook\" width=\"509\" height=\"138\" \/>\r\n<h6><\/h6>\r\nMost importantly, in this blog post we will show how easy it is to switch between MATLAB and Python code in your Jupyter notebook. Because we don\u2019t have to switch coding environment \u2013 we just switch kernels - this was the fastest model exchange prototype that we have created so far.\r\n<h6><\/h6>\r\nFor demonstration purposes, we made the example in this post lightweight and easy to follow. But you can extend this example to more complicated workflows, such as:\r\n<ul>\r\n \t<li>Processing and exploring domain-specific data (e.g., radar, wireless, audio, and medical images) in MATLAB for training a TensorFlow or PyTorch\u00ae model.<\/li>\r\n \t<li><a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/deep-learning-using-bayesian-optimization.html\">Using Bayesian optimization to train a model<\/a> in MATLAB and then, perform inference in TensorFlow or PyTorch.<\/li>\r\n \t<li>Create a TensorFlow or PyTorch model, and then <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/deep-learning-visualization-methods.html\">visualize the model behavior<\/a> in MATLAB.<\/li>\r\n \t<li>For more workflows that use MATLAB with Python together for AI, see our <a href=\"#other_blog_posts\">previous blog posts<\/a>\u00a0and\u00a0<a href=\"https:\/\/github.com\/matlab-deep-learning\">MATLAB Deep Learning GitHub<\/a>.<\/li>\r\n<\/ul>\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Set Up<\/strong><\/p>\r\n<a href=\"#other_blog_posts\">Previous blog posts<\/a> on the MATLAB Kernel for Jupyter showed how to use the kernel in Windows and Linux. In this blog post, we used a MacBook to execute the workflow. The initial setup happens at the macOS terminal. First, install the MATLAB Kernel for Jupyter.\r\n<h6><\/h6>\r\n<pre class=\"brush: python\" style=\"background-color: white;\">pip install jupyter-matlab-proxy\r\n<\/pre>\r\n<h6><\/h6>\r\nThe MATLAB executable is not necessarily on the system path (at least it was not on my Mac), so we run the following command.\r\n<h6><\/h6>\r\n<pre class=\"brush: python\" style=\"background-color: white;\">sudo ln -s \/Applications\/MATLAB_R2023a.app\/bin\/matlab  \/usr\/local\/bin\r\n<\/pre>\r\n<h6><\/h6>\r\nCheck that all the tools are installed as expected. After you verify that the right version of Python and all the necessary libraries are installed (and the MATLAB executable is on the path), open Jupyter notebook. There are other ways to start up a Jupyter notebook, for example by using CPython.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-12869 \" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/setup.png\" alt=\"Check Python and MATLAB installations, and then open Jupyter Notebook.\" width=\"735\" height=\"324\" \/>\r\n<h6><\/h6>\r\nNow, we are ready to run MATLAB and Python code from the same Jupyter notebook.\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Create Model with MATLAB Kernel<\/strong><\/p>\r\nFirst, we are going to create an <a href=\"https:\/\/www.mathworks.com\/discovery\/lstm.html\">LSTM<\/a> model in MATLAB. In your Jupyter notebook, specify your kernel as MATLAB. This is a one-click process.\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-12875 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/matlab_kernel.png\" alt=\"Change to MATLAB kernel.\" width=\"2367\" height=\"1517\" \/><\/h6>\r\n<h6><\/h6>\r\nYou can run the following MATLAB code the same way you would from any other MATLAB environment, for example from MATLAB desktop and MATLAB Online.\r\n<h6><\/h6>\r\n<strong>Create Layer Graph<\/strong>\r\n<h6><\/h6>\r\nCreate a long short-term memory (LSTM) network to classify sequence data. An LSTM network takes sequence data as input and makes predictions based on the individual time steps of the sequence data.\r\n<h6><\/h6>\r\n<pre>inputSize = 12;\r\nnumHiddenUnits = 100;\r\nnumClasses = 9;\r\n\r\nlayers = [\r\n       sequenceInputLayer(inputSize)\r\n       bilstmLayer(numHiddenUnits,OutputMode=\"last\")\r\n       fullyConnectedLayer(numClasses)\r\n       softmaxLayer];\r\n\r\nlgraph = layerGraph(layers);\r\n<\/pre>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<strong>Create Data Set<\/strong>\r\n<h6><\/h6>\r\nLoad the Japanese Vowels training data set. XTrain is a cell array containing 270 sequences of dimension 12 and varying length. YTrain is a categorical vector of labels \"1\",\"2\",...\"9\", which correspond to the nine speakers. To learn more about the data set, see <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/classify-sequence-data-using-lstm-networks.html\">Sequence Classification Using Deep Learning<\/a>.\r\n<h6><\/h6>\r\n<pre>[XTrain,YTrain] = japaneseVowelsTrainData;\r\n<\/pre>\r\n<h6><\/h6>\r\nPrepare the sequence data in XTrain for padding.\r\n<h6><\/h6>\r\n<pre>numObservations = numel(XTrain);\r\nfor i=1:numObservations\r\n    sequence = XTrain{i};\r\n    sequenceLengths(i) = size(sequence,2);\r\nend\r\n\r\n[sequenceLengths,idx] = sort(sequenceLengths);\r\nXTrain = XTrain(idx);\r\nYTrain = YTrain(idx);\r\n<\/pre>\r\n<h6><\/h6>\r\nPad XTrain along the second dimension.\r\n<h6><\/h6>\r\n<pre>XTrain = padsequences(XTrain,2);\r\n<\/pre>\r\n<h6><\/h6>\r\nPermute the sequence data from the Deep Learning Toolbox\u2122 ordering (CSN) to the TensorFlow ordering (NSC), where C is the number of features of the sequence, S is the sequence length, and N is the number of sequence observations. For more information on dimension ordering for different deep learning platforms and data types, see <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/tips-on-importing-models-from-tensorflow-pytorch-and-onnx.html#mw_ca5d4cba-9c12-4f01-8fe1-6329730c92b2\">Input Dimension Ordering<\/a>.\r\n<h6><\/h6>\r\n<pre>XTrain = permute(XTrain,[3,2,1]);\r\nYTrain = double(YTrain)-1;\r\n<\/pre>\r\n<h6><\/h6>\r\nSave the training data to a MAT file, so you can use them to train the exported TensorFlow network using Python code.\r\n<h6><\/h6>\r\n<pre>filename = \"training_data.mat\";\r\nsave(filename,\"XTrain\",\"YTrain\")\r\n<\/pre>\r\n<h6><\/h6>\r\nI wish the variables weren\u2019t lost when I switch between MATLAB and Python code in the Jupyter notebook. But it would be same if I was going to leave any MATLAB environment for a Python environment.\r\n<h6><\/h6>\r\n<strong>Convert Model from MATLAB to TensorFlow<\/strong>\r\n<h6><\/h6>\r\nExport the layer graph to TensorFlow. The <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/ref\/exportnetworktotensorflow.html\">exportNetworkToTensorFlow<\/a> function saves the TensorFlow model in the Python package myModel.\r\n<h6><\/h6>\r\n<pre>exportNetworkToTensorFlow(lgraph,\".\/myModel\")\r\n<\/pre>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Train Model with Python Kernel<\/strong><\/p>\r\nThen, we are going to train the exported TensorFlow model using Python. Specify your kernel as Python 3.\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-12893 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/python_kernel.png\" alt=\"Change to Python kernel.\" width=\"2355\" height=\"1460\" \/>\r\n<h6><\/h6>\r\nLoad the exported model from the Python package myModel.\r\n<h6><\/h6>\r\n<pre class=\"brush: python\" style=\"background-color: white;\">import myModel\r\nmodel = myModel.load_model()\r\nmodel.summary()\r\n<\/pre>\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-12899 \" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/model_summary.png\" alt=\"\" width=\"645\" height=\"338\" \/><\/h6>\r\n<h6><\/h6>\r\nLoad the training data in training_data.mat.\r\n<h6><\/h6>\r\n<pre class=\"brush: python\" style=\"background-color: white;\">import scipy.io as sio\r\ndata = sio.loadmat(\"training_data.mat\")\r\nXTrain = data[\"XTrain\"]\r\nYTrain = data[\"YTrain\"]\r\n<\/pre>\r\n<h6><\/h6>\r\nCompile and train the model.\r\n<h6><\/h6>\r\n<pre class=\"brush: python\" style=\"background-color: white;\">model.compile(optimizer = \"adam\", loss = \"sparse_categorical_crossentropy\", metrics=[\"accuracy\"])\r\nr = model.fit(XTrain, YTrain, epochs=100, batch_size=27)\r\n<\/pre>\r\n<h6><\/h6>\r\nSave the training history to a MAT file. I only need to do this because I am going to use the data in MATLAB in the next section.\r\n<pre class=\"brush: python\" style=\"background-color: white;\">sio.savemat(\"training_history.mat\",{\"training_history\":r.history})\r\n<\/pre>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Plot Metrics with MATLAB Kernel<\/strong><\/p>\r\nNow, we are switching back to MATLAB kernel to plot training metrics. We are going to create a very simple plot that you could create either with MATLAB or Python. For more complicated deep learning workflows and visualizations (for example, <a href=\"https:\/\/www.mathworks.com\/help\/vision\/ug\/semantic-segmentation-using-deep-learning.html\">semantic segmentation<\/a>), I find that MATLAB offers more options and easier to implement visualizations. Also, I don\u2019t need to install any additional Python libraries for plotting.\r\n<h6><\/h6>\r\nHowever, especially in this case, the point is that when it\u2019s so easy to switch between MATLAB and Python, why not just choose the most natural option for you.\r\n<h6><\/h6>\r\nLoad the training history.\r\n<h6><\/h6>\r\n<pre>load(\"training_history.mat\")\r\nhistory = struct(training_history)\r\n<\/pre>\r\n<h6><\/h6>\r\nPlot the loss and accuracy.\r\n<h6><\/h6>\r\n<pre>tiledlayout(2,1,TileSpacing=\"tight\")\r\n\r\nnexttile\r\nplot(history.accuracy)\r\nxlabel(\"Epochs\")\r\nylabel(\"Accuracy\")\r\n\r\nnexttile\r\nplot(history.loss)\r\nxlabel(\"Epochs\")\r\nylabel(\"Loss\")\r\n<\/pre>\r\n<h6><\/h6>\r\n<h6><img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-12908 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/accuracy_plot_screenshot.png\" alt=\"\" width=\"1690\" height=\"1419\" \/><\/h6>\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><strong>Conclusion<\/strong><\/p>\r\nCollaboration, integration, and easy access are key for developing AI-driven applications. On the AI blog, we have previously talked about how to and why use MATLAB with TensorFlow or PyTorch. The Jupyter integration makes it even easier to use different deep learning tools together, especially for prototyping and initial development. There is still a manual element in switching between kernels (meaning you can\u2019t run all the notebook cells), so we might consider <a href=\"https:\/\/www.mathworks.com\/products\/matlab\/matlab-and-python.html\">other options<\/a> to run MATLAB and Python code together for a plug-and-play version of the code.\r\n<h6><\/h6>\r\n&nbsp;\r\n<h6><\/h6>\r\n<p style=\"font-size: 18px;\"><a name=\"other_blog_posts\"><\/a><strong>Read More on MATLAB and Python Integration<\/strong><\/p>\r\nCheck out previous blog posts on how to use MATLAB with TensorFlow and PyTorch, and the release of MATLAB kernel for Jupyter:\r\n<ul>\r\n \t<li><a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2022\/12\/12\/quickly-investigate-pytorch-models-from-matlab\/\">Quickly Investigate PyTorch Models from MATLAB<\/a><\/li>\r\n \t<li><a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2022\/10\/04\/whats-new-in-interoperability-with-tensorflow-and-pytorch\/\">What\u2019s New in Interoperability with TensorFlow and PyTorch<\/a><\/li>\r\n \t<li><a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2022\/03\/18\/importing-models-from-tensorflow-pytorch-and-onnx\/\">Importing Models from TensorFlow, PyTorch, and ONNX<\/a><\/li>\r\n \t<li><a href=\"https:\/\/blog.tensorflow.org\/2023\/03\/tensorflow-with-matlab.html\">TensorFlow with MATLAB<\/a><\/li>\r\n \t<li><a href=\"https:\/\/blogs.mathworks.com\/matlab\/2023\/01\/30\/official-mathworks-matlab-kernel-for-jupyter-released\/\">Official MathWorks MATLAB kernel for Jupyter released<\/a><\/li>\r\n \t<li><a href=\"https:\/\/blogs.mathworks.com\/matlab\/2023\/06\/26\/matlab-kernel-for-jupyter-now-with-windows-support\/\">MATLAB kernel for Jupyter \u2013 Now with Windows support<\/a><\/li>\r\n<\/ul>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2023\/07\/pt_to_ml_inJupyter.png\" onError=\"this.style.display ='none';\" \/><\/div><p>\r\nThe MATLAB Kernel for Jupyter\u00ae (GitHub: jupyter-matlab-proxy) was released a few months ago. The MATLAB Kernel for Jupyter now supports Windows\u00ae, in addition to macOS\u00ae and Linux\u00ae. In this blog... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2023\/07\/26\/matlab-with-tensorflow-from-your-jupyter-notebook\/\">read more >><\/a><\/p>","protected":false},"author":194,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9,39,42],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/12857"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/194"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=12857"}],"version-history":[{"count":33,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/12857\/revisions"}],"predecessor-version":[{"id":15916,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/12857\/revisions\/15916"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=12857"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=12857"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=12857"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}