{"id":6931,"date":"2021-05-10T11:15:48","date_gmt":"2021-05-10T15:15:48","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=6931"},"modified":"2021-06-15T12:23:28","modified_gmt":"2021-06-15T16:23:28","slug":"semantic-segmentation-for-medical-imaging","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2021\/05\/10\/semantic-segmentation-for-medical-imaging\/","title":{"rendered":"Semantic Segmentation for Medical Imaging"},"content":{"rendered":"The following post is\u00a0by Dr.\u00a0Barath Narayanan<\/a>,\u00a0University of Dayton Research Institute\u00a0(UDRI) with co-authors: Dr.\u00a0Russell C. Hardie, and Redha Ali.<\/em>\r\n
<\/h6>\r\n\r\n\r\n\r\n
\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\nIn this blog, we apply Deep Learning based segmentation to skin lesions in dermoscopic images to aid in melanoma detection.\r\n
<\/h6>\r\nAffiliations:<\/strong>\r\n
<\/h6>\r\n*Sensors and Software Systems, University of Dayton Research Institute<\/a>, 300 College Park, Dayton, OH, 45469\r\n
<\/h6>\r\n**Department of Electrical and Computer Engineering, University of Dayton<\/a>, 300 College Park, Dayton, OH,\r\n45469\r\n
<\/h6>\r\n

Background<\/h2>\r\n

<\/h6>\r\nSkin lesion segmentation is an important step in Computer-Aided Diagnosis (CAD) of melanoma. In this blog, we present a Convolutional Neural Network (CNN) based segmentation approach applied to skin lesions in dermoscopic images. Early stage detection and diagnosis of melanoma detection increases one's survival rate significantly.\r\n
<\/h6>\r\nPlease cite the following article if you're using any part of the code for your research.\r\n
<\/h6>\r\n[1] Ali, R.<\/a>, Hardie, R. C.<\/a>, Narayanan, B. N.<\/a>, & De Silva, S. (2019, July). \"Deep learning ensemble methods for skin lesion analysis towards melanoma detection<\/a>\". In 2019 IEEE National Aerospace and Electronics Conference (NAECON) (pp. 311-316). IEEE.\r\n
<\/h6>\r\nDataset utilized for this blog is taken from ISIC 2018<\/a>. Instructions about the dataset are provided at the end of this post.\r\n
<\/h6>\r\n

Load the Dataset and Resize<\/h2>\r\n

<\/h6>\r\n\r\n\r\n\r\n
Raw images are loaded using imageDatastore<\/span>. It is a computationally efficient function to collect image information. Load the ground truth masks using pixelLabelDatastore<\/span>. White region in the ground truth mask indicates the \"lesion\" and rest of the image belongs to \"background\" class. The function pixelLabelImageDatastore<\/span> helps in tagging the raw image with its corresponding ground truth mask. Let's visualize certain random images from the dataset for our reference. Later, resize all images to a size 224 x 224 for the deep learning network.\r\n
<\/h6>\r\n<\/td>\r\n
\r\n\r\n
\"\"

Labeled image showing pixels as either background (black) and foreground \"lesion\" (white)<\/p><\/div><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

% Clear workspace <\/span>\r\nclear; close all; clc;\r\n\r\n% All images<\/span>\r\nimds=imageDatastore('ISIC2018_Task1-2_Training_Input','IncludeSubfolders',true);\r\n\r\n% Define class names and their corresponding IDs<\/span>\r\nclassNames=[\"Lesion\",\"Background\"];\r\nlabelIDs=[255,0];\r\n\r\n% Create a pixelLabelDatastore holding the ground truth pixel labels<\/span>\r\npxds=pixelLabelDatastore('ISIC2018_Task1_Training_GroundTruth',classNames, labelIDs);\r\n\r\n% Create a pixel label image datastore of all images  <\/span>\r\npximds=pixelLabelImageDatastore(imds,pxds);\r\n\r\n% Number of Images<\/span>\r\ntotal_num_images=length(pximds.Images);\r\n\r\n% Visualize random images<\/span>\r\nperm=randperm(total_num_images,4);\r\n\r\nfigure;\r\n% Visualize the images with Mask<\/span>\r\nfor idx=1:length(perm)\r\n    \r\n    [~,filename]=fileparts(pximds.Images{idx});\r\n    subplot(2,2,idx);\r\n    imshow(imread(pximds.Images{perm(idx)}));\r\n    hold on;\r\n    visboundaries(imread(pximds.PixelLabelData{perm(idx)}),'Color','r');\r\n    title(sprintf('%s',filename),'Interpreter',\"none\");\r\nend<\/pre>\r\n\"\"\r\n
% Desired Image Size <\/span>\r\nimageSize=[224 224 3];\r\n\r\n% Create a pixel label image datastore of all resized images  <\/span>\r\npximdsResz=pixelLabelImageDatastore(imds,pxds,'OutputSize',imageSize);\r\n\r\n% Clear all variables except the necessary variables<\/span>\r\nclearvars -except pximdsResz classNames total_num_images imageSize\r\n\r\n<\/pre>\r\n
Split the Dataset - Training, Validation and Testing<\/h2>\r\n
% Randomly select 100 images for testing from the dataset<\/span>\r\ntest_idx=randperm(total_num_images,100);\r\n\r\n% Rest of the indices are utilize for training and validation <\/span>\r\ntrain_valid_idx=setdiff(1:total_num_images,test_idx);\r\n\r\n% Randomly pick 100 images for validation from the training dataset<\/span>\r\nvalid_idx=train_valid_idx(randperm(length(train_valid_idx),100));\r\n\r\n% Rest of the indices are used for training<\/span>\r\ntrain_idx=setdiff(train_valid_idx,valid_idx);\r\n\r\n% Train Dataset<\/span>\r\npximdsTrain=partitionByIndex(pximdsResz,train_idx);\r\n\r\n% Validation Dataset<\/span>\r\npximdsValid=partitionByIndex(pximdsResz,valid_idx);\r\n\r\n% Test Dataset<\/span>\r\npximdsTest=partitionByIndex(pximdsResz,test_idx);<\/pre>\r\n
<\/h6>\r\n
Deep Learning Approach<\/h2>\r\n
<\/h6>\r\nDefine the CNN network for training the network along with the parameters necessary.\r\n
<\/h6>\r\nIn this blog, we study the performance using DeepLab v3+ network. DeepLab v3+ is a CNN for semantic image segmentation. It utilizes an encoder-decoder based architecture with dilated convolutions and skip convolutions to segment images. In [1], we present an ensemble approach of combining both U-Net with DeepLab v3+ network. In the blog, we solely focus on DeepLab v3+ network using ResNet50 architecture. Feel free to change the hyperparameters and observe the performance.\r\n
<\/h6>\r\nNotes:<\/strong>\r\n
<\/h6>\r\n