{"id":15083,"date":"2024-05-15T09:01:27","date_gmt":"2024-05-15T13:01:27","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=15083"},"modified":"2024-05-29T14:50:16","modified_gmt":"2024-05-29T18:50:16","slug":"data-type-conversion-between-matlab-and-python-whats-new-in-r2024a","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2024\/05\/15\/data-type-conversion-between-matlab-and-python-whats-new-in-r2024a\/","title":{"rendered":"Data Type Conversion Between MATLAB and Python: What\u2019s New in R2024a"},"content":{"rendered":"
<\/h6>\r\nWhen combining MATLAB with Python\u00ae to create deep learning workflows, data type conversion between the two frameworks can be time consuming and sometimes perplexing. I 've certainly experimented with figuring out how to make MATLAB data compatible with a Python-based model and vice versa. So, you can understand why I am so excited that there are two new data type conversions introduced in MATLAB R2024a.\r\n
<\/h6>\r\n\"Conversion\r\n
<\/h6>\r\n \r\n
<\/h6>\r\nIn the following table you can see the new data type conversions. And it came as no surprise to me that they made the list of Mike Croucher's favorite R2024a updates<\/a>.\r\n
<\/h6>\r\n\r\n\r\n\r\n\r\n\r\n\r\n
Python Data Type<\/strong><\/td>\r\n<\/td>\r\nMATLAB Data Type<\/strong><\/td>\r\n<\/tr>\r\n
Pandas DataFrame<\/td>\r\n\"\"<\/td>\r\nMATLAB table<\/a><\/td>\r\n<\/tr>\r\n
Python dictionary<\/td>\r\n\"\"<\/td>\r\nMATLAB dictionary<\/a><\/td>\r\n<\/tr>\r\n
Python dictionary<\/td>\r\n\"\"<\/td>\r\nMATLAB structure<\/a><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\nPerforming the data type conversions is very easy and you can learn all the details in these documentation topics: Use Python Pandas DataFrames in MATLAB<\/a> and Use Python Dictionaries in MATLAB<\/a>. So, in this blog post, instead of showing you the code for all the possible conversions, I am going to walk you through an object detection example and present a use case for converting data from a Python dictionary to a MATLAB structure.\r\n
<\/h6>\r\n \r\n
<\/h6>\r\n

Object Detection Example<\/strong><\/p>\r\nThis example calls a PyTorch model from MATLAB to detect objects in the input image. When you call the PyTorch model you want to (1) pass MATLAB data to a format that the model can process and (2) convert the model outputs (Python data type) to a MATLAB data type that can be used in MATLAB for visualization, as shown here, but also as an input to the next steps in your workflow.\r\n

<\/h6>\r\n

Python Environment<\/strong><\/p>\r\nSet up the Python interpreter for MATLAB by using the pyenv<\/a>\u00a0function.\r\n

pe = pyenv(Version=\".\\env\\Scripts\\python.exe\",ExecutionMode=\"OutOfProcess\");\r\n<\/pre>\r\n
<\/h6>\r\n
Python Code for Object Detection<\/strong><\/p>\r\nThe following Python code is saved in the PT_object_detection.py file, which you can call from MATLAB to perform object detection with a PyTorch model.\r\n
import torch\r\nimport torchvision as vision\r\nimport numpy\r\n\r\ndef loadPTmodel():\r\n    # Initialize model with the best available weights\r\n    weights = vision.models.detection.FasterRCNN_ResNet50_FPN_V2_Weights.DEFAULT\r\n    model = vision.models.detection.fasterrcnn_resnet50_fpn_v2(weights=weights,box_score_thresh=0.95)\r\n    model.eval()\r\n\r\n    return model, weights\r\n\r\ndef detectPT(img,model,weights):\r\n    # Reshape image and convert to a tensor.\r\n    X = numpy.asarray(img)\r\n    X_torch1 = torch.from_numpy(numpy.copy(X))\r\n    if X_torch1.ndim==3:\r\n      X_torch = torch.permute(X_torch1,(2,0,1))\r\n    elif X_torch1.ndim==4:\r\n      X_torch = torch.permute(X_torch1,(3,2,0,1))\r\n    # Initialize the inference transforms\r\n    preprocess = weights.transforms()\r\n    # Apply inference preprocessing transforms\r\n    batch = [preprocess(X_torch)]\r\n    # Use the model \r\n    if X_torch.ndim==3:\r\n      prediction = model(batch)[0]\r\n    elif X_torch.ndim==4:\r\n      prediction = model(list(batch[0]))\r\n\r\n    return prediction\r\n<\/pre>\r\n
<\/h6>\r\n
Object Detection<\/strong><\/p>\r\nRead the image that you want to use for object detection.\r\n
img_filename = \"testCar.png\";\r\nimg = imread(img_filename);\r\n<\/pre>\r\n
<\/h6>\r\nPerform object detection with a PyTorch model using co-execution.\r\n
pyrun(\"from PT_object_detection import loadPTmodel, detectPT\")\r\n[model,weights] = pyrun(\"[a,b] = loadPTmodel()\",[\"a\" \"b\"]);\r\npredictions_pt = pyrun(\"a = detectPT(b,c,d)\",\"a\",b=img,c=model,d=weights);\r\n<\/pre>\r\n
<\/h6>\r\nThe output of the object detector is a Python dictionary.\r\n
class(predictions_pt)\r\n<\/pre>\r\n
ans = 'py.dict'\r\n<\/pre>\r\n
<\/h6>\r\n
Convert Predictions<\/strong><\/p>\r\nConvert the objection detection results from a Python dictionary to a MATLAB structure.\r\n
predictions = struct(predictions_pt)\r\n<\/pre>\r\n
predictions = struct with fields:\r\n     boxes: [1\u00d71 py.torch.Tensor]\r\n    labels: [1\u00d71 py.torch.Tensor]\r\n    scores: [1\u00d71 py.torch.Tensor]\r\n<\/pre>\r\n
<\/h6>\r\nThe data variables in the structure prediction are PyTorch tensors. Convert the variables into MATLAB arrays.\r\n
predictions.boxes = double(predictions.boxes.detach().numpy);\r\npredictions.labels = double(predictions.labels.tolist)';\r\npredictions.scores = double(predictions.scores.tolist)';\r\npredictions\r\n<\/pre>\r\n
predictions = struct with fields:\r\n     boxes: [4\u00d74 double]\r\n    labels: [4\u00d71 double]\r\n    scores: [4\u00d71 double]\r\n<\/pre>\r\n
<\/h6>\r\nThe bounding boxes require further processing to align with the input image.\u00a0 A bounding box is an axis-aligned rectangle defined in spatial coordinates as an M<\/em>x4 numeric matrix with rows of the form [x y w h<\/em>], where:\r\n