{"id":4850,"date":"2020-08-20T09:55:43","date_gmt":"2020-08-20T13:55:43","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=4850"},"modified":"2021-04-06T15:45:51","modified_gmt":"2021-04-06T19:45:51","slug":"diabetic-retinopathy-detection","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/08\/20\/diabetic-retinopathy-detection\/","title":{"rendered":"Diabetic Retinopathy Detection"},"content":{"rendered":"Post by Dr. <a href=\"https:\/\/udayton.edu\/directory\/udri\/sensorsoftwaresystems\/narayanan-barath.php\">Barath Narayanan<\/a>, <a href=\"https:\/\/udayton.edu\/udri\/\">University of Dayton Research Institute<\/a> (UDRI) with co-authors: Dr. <a href=\"https:\/\/udayton.edu\/directory\/engineering\/electrical_and_computer\/hardie_russell.php\">Russell C. Hardie<\/a>, University of Dayton (UD), <a href=\"https:\/\/www.linkedin.com\/in\/mssdesilva\/\">Manawduge Supun De Silva<\/a>, UD, and <a href=\"https:\/\/www.linkedin.com\/in\/the-real-nathan-kueterman\">Nathaniel K. Kueterman<\/a>, UD.\r\n<h6><\/h6>\r\n<h2>Introduction<\/h2>\r\n<h6><\/h6>\r\nDiabetic Retinopathy (DR) is one of the leading cause for blindness, affecting over 93 million people across the world. DR is an eye disease associated with diabetes. Detection and grading DR at an early stage would help in preventing permanent vision loss. Automated detection and grading during the retinal screening process would help in providing a valuable second opinion. In this blog, we implement a simple transfer-learning based approach using a deep Convolutional Neural Network (CNN) to <strong>detect DR<\/strong>.\r\n<h6><\/h6>\r\nPlease cite the following articles if you're using any part of the code for your research:\r\n<ul>\r\n \t<li>Narayanan, B. N., Hardie, R. C., De Silva, M. S., &amp; Kueterman, N. K. (2020). <a href=\"https:\/\/doi.org\/10.1117\/1.JMI.7.3.034501\">Hybrid machine learning architecture for automated detection and grading of retinal images for diabetic retinopathy<\/a>. Journal of Medical Imaging, 7(3), 034501.<\/li>\r\n \t<li>Narayanan, B. N., De Silva, M. S., Hardie, R. C., Kueterman, N. K., &amp; Ali, R. (2019). <a href=\"https:\/\/arxiv.org\/abs\/1912.09621\">Understanding Deep Neural Network Predictions for Medical Imaging Applications. arXiv preprint arXiv:1912.09621<\/a>.<\/li>\r\n<\/ul>\r\nThe Kaggle blindness detection challenge dataset (<a href=\"https:\/\/www.kaggle.com\/c\/aptos2019-blindness-detection\">APTOS 2019 Dataset<\/a>) contains separate training and testing cases. In this blog, we solely utilize the training dataset to study and estimate the performance. These images were captured at the Aravind Eye Hospital, India. The training dataset contains 3662 images marked into different categories (Normal, Mild DR, Moderate DR, Severe DR, and Proliferative DR) by expert clinicians. <em>Note that, in this blog, we solely focus on detecting DR, you could find more details about our grading architecture in our <a href=\"https:\/\/doi.org\/10.1117\/1.JMI.7.3.034501\">paper<\/a>.<\/em>\r\n<h6><\/h6>\r\n<h2>Grouping Data by Category<\/h2>\r\nWe extract the labels from excel sheet and segregate the images into 2-folders as 'no' or 'yes' as we're solely focused on detecting DR in this blog. <em>The helper code for splitting the data into categories is at the end of this post. <\/em>\r\n<h6><\/h6>\r\n<h2>Load the Database<\/h2>\r\n<h6><\/h6>\r\nLet's begin by loading the database using <a href=\"https:\/\/www.mathworks.com\/help\/matlab\/ref\/matlab.io.datastore.imagedatastore.html\">imageDatastore<\/a>. It's a computationally efficient function to load the images along with its labels for analysis.\r\n<h6><\/h6>\r\n<pre><span class=\"comment\">% Two-class Datapath<\/span>\r\ntwo_class_datapath = 'Train Dataset Two Classes';\r\n\r\n<span class=\"comment\">% Image Datastore<\/span>\r\nimds=imageDatastore(two_class_datapath, ...\r\n    'IncludeSubfolders',true, ...\r\n    'LabelSource','foldernames');\r\n\r\n<span class=\"comment\">% Determine the split up<\/span>\r\ntotal_split=countEachLabel(imds)\r\n<\/pre>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"479\" height=\"134\" class=\"alignnone size-full wp-image-4858\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/second_1.png\" alt=\"\" \/>\r\n<h2>Visualize the Images<\/h2>\r\nLet's visualize the images and see how images differ for each class. It would also help us determine the type of classification technique that could be applied for distinguishing the two classes. Based on the images, we could identify preprocessing techniques that would assist our classification process. We could also determine the type of CNN architecture that could be utilized for the study based on the similarities within the class and differences across classes. In this article, we implement transfer learning using inception-v3 architecture. You can <a href=\"https:\/\/doi.org\/10.1117\/1.JMI.7.3.034501\">read<\/a> our paper to see the performance of different preprocessing operations and other established architectures.\r\n<h6><\/h6>\r\n<pre><span class=\"comment\">% Number of Images<\/span>\r\nnum_images=length(imds.Labels);\r\n\r\n<span class=\"comment\">% Visualize random 20 images<\/span>\r\nperm=randperm(num_images,20);\r\nfigure;\r\nfor idx=1:20\r\n    \r\n    subplot(4,5,idx);\r\n    imshow(imread(imds.Files{perm(idx)}));\r\n    title(sprintf('%s',imds.Labels(perm(idx))))\r\n    \r\nend<\/pre>\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"898\" height=\"692\" class=\"alignnone size-full wp-image-4884\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/retinopathy_2.png\" alt=\"\" \/>\r\n<h6><\/h6>\r\n<h2>Training, Testing and Validation<\/h2>\r\nLet\u2019s split the dataset into training, validation and testing. At first, we are splitting the dataset into groups of 80% (training &amp; validation) and 20% (testing). Make sure to split equal quantity of each class.\r\n<h6><\/h6>\r\n<pre><span class=\"comment\">% Split the Training and Testing Dataset<\/span>\r\ntrain_percent=0.80;\r\n[imdsTrain,imdsTest]=splitEachLabel(imds,train_percent,'randomize');\r\n \r\n<span class=\"comment\">% Split the Training and Validation<\/span>\r\nvalid_percent=0.1;\r\n[imdsValid,imdsTrain]=splitEachLabel(imdsTrain,valid_percent,'randomize');\r\n<\/pre>\r\nThis leaves us with the following counts:\r\n\r\n&nbsp;\r\n<table style=\"height: 134px;\" width=\"250\">\r\n<tbody>\r\n<tr>\r\n<td><\/td>\r\n<td><strong>Yes<\/strong><\/td>\r\n<td><strong>No<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td>Training Set:<\/td>\r\n<td><strong>1337<\/strong><\/td>\r\n<td><strong>1300<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td>Validation Set:<\/td>\r\n<td><strong>144<\/strong><\/td>\r\n<td><strong>149<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td>Test Set:<\/td>\r\n<td><strong>361<\/strong><\/td>\r\n<td><strong>371<\/strong><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h6><\/h6>\r\n<h2>Deep Learning Approach<\/h2>\r\nLet\u2019s adopt a transfer learning approach to classify retinal images. In this article, I\u2019m utilizing Inception-v3 for classification, you could utilize other transfer learning approaches as mentioned in the <a href=\"https:\/\/doi.org\/10.1117\/1.JMI.7.3.034501\">paper<\/a> or any other architecture that you think might be suited for this application. My MathWorks blogs on transfer learning using other established networks can be found here: <a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2019\/11\/14\/deep-learning-for-medical-imaging-malaria-detection\/\">AlexNet<\/a>, <a href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/03\/18\/deep-learning-for-medical-imaging-covid-19-detection\/\">ResNet<\/a>\r\n<h6><\/h6>\r\n<h2>Training<\/h2>\r\nWe will utilize validation patience of 3 as the stopping criteria. For starters, we use 'MaxEpochs' as 2 for our training, but we can tweak it further based on our training progress. Ideally, we want the validation performance to be high when training process is stopped. We choose a mini-batch size of 32 based on our hardware memory constraints, you could pick a bigger mini-batch size but make sure to change the other parameters accordingly.\r\n<h6><\/h6>\r\n<pre><span class=\"comment\">% Converting images to 299 x 299 to suit the architecture<\/span>\r\naugimdsTrain = augmentedImageDatastore([299 299],imdsTrain);\r\naugimdsValid = augmentedImageDatastore([299 299],imdsValid);\r\n\r\n<span class=\"comment\">% Set the training options<\/span>\r\noptions = trainingOptions('adam','MaxEpochs',2,'MiniBatchSize',32,...\r\n'Plots','training-progress','Verbose',0,'ExecutionEnvironment','parallel',...\r\n'ValidationData',augimdsValid,'ValidationFrequency',50,'ValidationPatience',3);\r\n\r\nnetTransfer = trainNetwork(augimdsTrain,incepnet,options);<\/pre>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"1002\" height=\"519\" class=\"alignnone size-full wp-image-4886\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/retinopathy_3.png\" alt=\"\" \/>\r\n<h6><\/h6>\r\n<h2>Testing and Performance Evaluation<\/h2>\r\n<pre><span class=\"comment\">% Reshape the test images match with the network <\/span>\r\naugimdsTest = augmentedImageDatastore([299 299],imdsTest);\r\n\r\n<span class=\"comment\">% Predict Test Labels<\/span>\r\n[predicted_labels,posterior] = classify(netTransfer,augimdsTest);\r\n\r\n<span class=\"comment\">% Actual Labels<\/span>\r\nactual_labels = imdsTest.Labels;\r\n\r\n<span class=\"comment\">% Confusion Matrix<\/span>\r\nfigure\r\nplotconfusion(actual_labels,predicted_labels)\r\ntitle('Confusion Matrix: Inception v3');<\/pre>\r\n<h6><\/h6>\r\n<img decoding=\"async\" loading=\"lazy\" class=\"alignnone wp-image-4926 size-full\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/retinopathy_4a.png\" alt=\"\" width=\"450\" height=\"450\" \/>\r\n<h6><\/h6>\r\n<pre><span class=\"comment\">% ROC Curve<\/span>\r\ntest_labels=double(nominal(imdsTest.Labels));\r\n[fp_rate,tp_rate,T,AUC] = perfcurve(test_labels,posterior(:,2),2);\r\nfigure;\r\nplot(fp_rate,tp_rate,'b-');hold on;\r\ngrid on;\r\nxlabel('False Positive Rate');\r\nylabel('Detection Rate');<\/pre>\r\n<img decoding=\"async\" loading=\"lazy\" width=\"594\" height=\"451\" class=\"alignnone size-large wp-image-4892\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/retinopathy_5.png\" alt=\"\" \/>\r\n<h2>Class Activation Mapping Results<\/h2>\r\n<h6><\/h6>\r\nWe visualize the Class Activation Mapping (CAM) results for these networks for different DR cases using the code: <a href=\"https:\/\/www.mathworks.com\/help\/deeplearning\/examples\/investigate-network-predictions-using-class-activation-mapping.html\">https:\/\/www.mathworks.com\/help\/deeplearning\/examples\/investigate-network-predictions-using-class-activation-mapping.html<\/a>. This would help in providing insights behind the algorithm's decision to the doctors.\r\n<h6><\/h6>\r\nHere are the results obtained for various cases:\r\n\r\n<img decoding=\"async\" loading=\"lazy\" width=\"725\" height=\"311\" class=\"alignnone size-full wp-image-4894\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/retinopathy_6.png\" alt=\"\" \/>\r\n<h2>Conclusions<\/h2>\r\nIn this blog, we have presented a simple deep learning-based classification approach for CAD of DR in retinal images. The classification algorithm using Inception-v3 without any preprocessing performed relatively well with an overall accuracy of 98.0% and an AUC of 0.9947 (results may vary because of the random split). In the <a href=\"https:\/\/doi.org\/10.1117\/1.JMI.7.3.034501\">paper<\/a>, we studied the performance of various established CNN architectures for the same set of training and testing cases under different preprocessing conditions. Combining the results of various architectures provides a boost in performance both in terms of AUC and overall accuracy. A comprehensive study of these algorithms, both in terms of computation (memory and time) and performance, allows the subject matter experts to make an informed choice. In addition, we have presented our novel architecture approaches in the <a href=\"https:\/\/doi.org\/10.1117\/1.JMI.7.3.034501\">paper<\/a> for detection and grading of DR.\r\n<h6><\/h6>\r\n<h2>About the Author<\/h2>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td><img decoding=\"async\" loading=\"lazy\" width=\"1024\" height=\"683\" class=\"alignnone size-large wp-image-3293\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2019\/10\/image_bio-1024x683.jpg\" alt=\"\" \/><\/td>\r\n<td style=\"padding: 10px;\">Dr. <a href=\"https:\/\/udayton.edu\/directory\/udri\/sensorsoftwaresystems\/narayanan-barath.php\">Barath Narayanan<\/a> graduated with MS and Ph.D. degree in Electrical Engineering from the University of Dayton (UD) in 2013 and 2017 respectively. He currently holds a joint appointment as a Research Scientist at <a href=\"https:\/\/udayton.edu\/udri\/capabilities\/software\/index.php\">UDRI's Software Systems Group<\/a> and as an Adjunct Faculty for the <a href=\"https:\/\/udayton.edu\/engineering\/departments\/electrical_and_computer\/index.php\">ECE department<\/a> at UD. His research interests include deep learning, machine learning, computer vision, and pattern recognition.<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<h2>Helper Code<\/h2>\r\n<h6><\/h6>\r\n<h3>Code for grouping data by DR category (yes or no)<\/h3>\r\n<h6><\/h6>\r\nAfter downloading the ZIP files from the website and extracting them to a folder called \"train_images\". Make sure to download the excel sheet (train.csv - convert it to .xlsx for this code) containing the true labels by expert clinicians. We extract the labels from excel sheet and segregate the images into 2-folders as 'no' or 'yes' as we solely focus on detecting DR in this blog.\r\n<h6><\/h6>\r\n<pre><span class=\"comment\">% Training Data path<\/span>\r\ndatapath='train_images\\';\r\n\r\n<span class=\"comment\">% Two-class Data path<\/span>\r\ntwo_class_datapath='Train Dataset Two Classes\\';\r\n\r\n<span class=\"comment\">% Class Names<\/span>\r\nclass_names={'No','Yes'};\r\nmkdir(sprintf('%s%s',two_class_datapath,class_names{1}))\r\nmkdir(sprintf('%s%s',two_class_datapath,class_names{2}))\r\n\r\n<span class=\"comment\">% Read the Excel Sheet with Labels<\/span>\r\n[num_data,text_data]=xlsread('train.xlsx');\r\n\r\n<span class=\"comment\">% Determine the Labels<\/span>\r\ntrain_labels=num_data(:,1);\r\n\r\n<span class=\"comment\">% Merge all labels marked into Mild, Medium, Severe and Proliferative DR <\/span>\r\n<span class=\"comment\">% into a single category 'Yes' <\/span>\r\ntrain_labels(train_labels~=0)=2;\r\n\r\n<span class=\"comment\">% Rest of the dataset belongs to 'No' category<\/span>\r\ntrain_labels(train_labels==0)=1;\r\n\r\n<span class=\"comment\">% Filename<\/span>\r\nfilename=text_data(2:end,1);\r\n\r\n<span class=\"comment\">% Now, write these images 2-folders 'Yes' or 'No' for us to develop a deep<\/span>\r\n<span class=\"comment\">% learning architecture utilizing Deep learning toolbox<\/span>\r\n<span class=\"comment\">% Determine the Files put them in separate folder<\/span>\r\nfor idx=1:length(filename)\r\n    \r\n   <span class=\"comment\">% You could uncomment if you would like to see live progress<\/span>\r\n    %  fprintf('Processing %d among %d files:%s \\n',idx,length(filename),filename{idx})[\/%]\r\n    \r\n    <span class=\"comment\">% Read the image<\/span>\r\n    current_filename=strrep(filename{idx}, char(39), '');\r\n    img=imread(sprintf('%s%s.png',datapath,current_filename));\r\n    \r\n    <span class=\"comment\">% Write the image in the respective folder<\/span>\r\n    imwrite(img,sprintf('%s%s%s%s.png',two_class_datapath,class_names{train_labels(idx)},'\\',current_filename));\r\n    \r\n    clear img;\r\n    \r\nend\r\n\r\n<\/pre>","protected":false},"excerpt":{"rendered":"<div class=\"overview-image\"><img decoding=\"async\"  class=\"img-responsive\" src=\"https:\/\/blogs.mathworks.com\/deep-learning\/files\/2020\/08\/second_1.png\" onError=\"this.style.display ='none';\" \/><\/div><p>Post by Dr. Barath Narayanan, University of Dayton Research Institute (UDRI) with co-authors: Dr. Russell C. Hardie, University of Dayton (UD), Manawduge Supun De Silva, UD, and Nathaniel K.... <a class=\"read-more\" href=\"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/08\/20\/diabetic-retinopathy-detection\/\">read more >><\/a><\/p>","protected":false},"author":156,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4850"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/156"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=4850"}],"version-history":[{"count":25,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4850\/revisions"}],"predecessor-version":[{"id":4930,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4850\/revisions\/4930"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=4850"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=4850"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=4850"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}