{"id":4043,"date":"2020-03-18T13:01:49","date_gmt":"2020-03-18T13:01:49","guid":{"rendered":"https:\/\/blogs.mathworks.com\/deep-learning\/?p=4043"},"modified":"2021-04-06T15:48:44","modified_gmt":"2021-04-06T19:48:44","slug":"deep-learning-for-medical-imaging-covid-19-detection","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/deep-learning\/2020\/03\/18\/deep-learning-for-medical-imaging-covid-19-detection\/","title":{"rendered":"Deep Learning for Medical Imaging: COVID-19 Detection"},"content":{"rendered":"\r\n\r\n\r\n
I'm pleased to publish another post from Barath Narayanan<\/a>, University of Dayton Research Institute<\/a> (UDRI), LinkedIn Profile<\/a>.\r\n
<\/h6>\r\nCo-author: Dr. Russell C. Hardie<\/a>, University of Dayton (UD)\r\n
<\/h6>\r\nDr. Barath Narayanan <\/a>graduated with MS and Ph.D. degree in Electrical Engineering from the University of Dayton (UD) in 2013 and 2017 respectively. He currently holds a joint appointment as a Research Scientist at UDRI's Software Systems Group<\/a> and as an Adjunct Faculty for the ECE department<\/a> at UD. His research interests include deep learning, machine learning, computer vision, and pattern recognition.<\/td>\r\n
\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\nIn this blog, we are applying a Deep Learning (DL) based technique for detecting COVID-19 on Chest Radiographs using MATLAB.<\/span>\r\n
<\/h6>\r\n

Background<\/h2>\r\n

<\/h6>\r\nCoronavirus disease<\/a> (COVID-19) is a new strain of disease in humans discovered in 2019 that has never been identified in the past. Coronavirus is a large family of viruses that causes illness in patients ranging from common cold to advanced respiratory syndromes such as Middle East Respiratory Syndrome (MERS-COV) and Severe Acute Respiratory Syndrome (SARS-COV). Many people are currently affected and are being treated across the world causing a global pandemic. In the United States alone, 160 million to 214 million people could be infected over the course of the COVID-19 epidemic (https:\/\/www.nytimes.com\/2020\/03\/13\/us\/coronavirus-deaths-estimate.html<\/a>). Several countries have declared a national emergency and have quarantined millions of people. Here is a detailed article on how coronavirus affects people: https:\/\/www.nytimes.com\/article\/coronavirus-body-symptoms.html<\/a>\r\n
<\/h6>\r\nDetection and diagnosis tools <\/strong>offer a valuable second opinion to the doctors and assist them in the screening process. This type of mechanism would also assist in providing results to the doctors quickly. In this blog, we are applying a Deep Learning (DL) based technique for detecting COVID-19 on Chest Radiographs using MATLAB.\r\n
<\/h6>\r\nThe COVID-19 dataset<\/a> utilized in this blog was curated by Dr. Joseph Cohen<\/a>, a postdoctoral fellow at the University of Montreal. Thanks to the article<\/a> by Dr. Adrian Rosebrock<\/a> for making this chest radiograph dataset reachable to researchers across the globe and for presenting the initial work using DL. Note that we solely utilize the x-ray images. You should be able to download the images from the article directly. After downloading the ZIP files from the website and extracting them to a folder called \"Covid 19\", we have one sub-folder per class in \"dataset\". Label \"Covid\" indicates the presence of COVID-19 in the patient and \"normal\" otherwise. Since, we have equal distribution (25 images) of both classes, there is no class imbalance issue here.\r\n
<\/h6>\r\n

Load the Dataset<\/h2>\r\n

<\/h6>\r\nLet's begin by loading the database using imageDatastore<\/span>. It is a computationally efficient function to load the images along with its labels for analysis.\r\n
<\/h6>\r\n
% Clear workspace<\/span>\r\nclear; close all; clc;\r\n \r\n% Images Datapath \u2013 Please modify your path accordingly <\/span>\r\ndatapath='dataset';\r\n \r\n% Image Datastore<\/span>\r\nimds=imageDatastore(datapath, ...\r\n    'IncludeSubfolders',true, ...\r\n    'LabelSource','foldernames');\r\n% Determine the split up<\/span>\r\ntotal_split=countEachLabel(imds)<\/pre>\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\n
Visualize the Images<\/h2>\r\nLet's visualize the images and see how images differ for each class. It would also help us determine the type of classification technique that could be applied for distinguishing the two classes. Based on the images, we could identify preprocessing techniques that would assist our classification process. We could also determine the type of CNN architecture that could be utilized for the study based on the similarities within the class and differences across classes.\r\n
<\/h6>\r\n
% Number of Images<\/span>\r\nnum_images=length(imds.Labels);\r\n\r\n% Visualize random images<\/span>\r\nperm=randperm(num_images,6);\r\nfigure;\r\nfor idx=1:length(perm)\r\n    \r\n    subplot(2,3,idx);\r\n    imshow(imread(imds.Files{perm(idx)}));\r\n    title(sprintf('%s',imds.Labels(perm(idx))))\r\n    \r\nend\r\n<\/pre>\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\nIn order to ease the classification process for our DL architecture, we apply simple preprocessing techniques and the function is enclosed at the end of this article.\r\n
<\/h6>\r\n
K-fold Validation<\/h2>\r\nAs you already know that there is a limited set of images available in this dataset, we split the dataset into 10-folds for analysis i.e. 10 different algorithms would be trained using different set of images from the dataset. This type of validation study would provide us a better estimate of our performance in comparison to typical hold-out validation method.\r\n
<\/h6>\r\nWe adopt ResNet-50 architecture in this blog as it has proven to be highly effective for various medical imaging applications [1,2].\r\n
<\/h6>\r\n
% Number of folds<\/span>\r\nnum_folds=10;\r\n\r\n% Loop for each fold<\/span>\r\nfor fold_idx=1:num_folds\r\n    \r\n    fprintf('Processing %d among %d folds \\n',fold_idx,num_folds);\r\n    \r\n   % Test Indices for current fold<\/span>\r\n    test_idx=fold_idx:num_folds:num_images;\r\n\r\n    % Test cases for current fold<\/span>\r\n    imdsTest = subset(imds,test_idx);\r\n    \r\n    % Train indices for current fold<\/span>\r\n    train_idx=setdiff(1:length(imds.Files),test_idx);\r\n    \r\n    % Train cases for current fold<\/span>\r\n    imdsTrain = subset(imds,train_idx);\r\n \r\n    % ResNet Architecture <\/span>\r\n    net=resnet50;\r\n    lgraph = layerGraph(net);\r\n    clear net;\r\n    \r\n    % Number of categories<\/span>\r\n    numClasses = numel(categories(imdsTrain.Labels));\r\n    \r\n    % New Learnable Layer<\/span>\r\n    newLearnableLayer = fullyConnectedLayer(numClasses, ...\r\n        'Name','new_fc', ...\r\n        'WeightLearnRateFactor',10, ...\r\n        'BiasLearnRateFactor',10);\r\n    \r\n    % Replacing the last layers with new layers<\/span>\r\n    lgraph = replaceLayer(lgraph,'fc1000',newLearnableLayer);\r\n    newsoftmaxLayer = softmaxLayer('Name','new_softmax');\r\n    lgraph = replaceLayer(lgraph,'fc1000_softmax',newsoftmaxLayer);\r\n    newClassLayer = classificationLayer('Name','new_classoutput');\r\n    lgraph = replaceLayer(lgraph,'ClassificationLayer_fc1000',newClassLayer);\r\n    \r\n    \r\n    % Preprocessing Technique<\/span>\r\n    imdsTrain.ReadFcn = @(filename)preprocess_Xray(filename);\r\n    imdsTest.ReadFcn = @(filename)preprocess_Xray(filename);\r\n    \r\n    % Training Options, we choose a small mini-batch size due to limited images <\/span>\r\n    options = trainingOptions('adam',...\r\n        'MaxEpochs',30,'MiniBatchSize',8,...\r\n        'Shuffle','every-epoch', ...\r\n        'InitialLearnRate',1e-4, ...\r\n        'Verbose',false, ...\r\n        'Plots','training-progress');\r\n    \r\n    % Data Augumentation<\/span>\r\n    augmenter = imageDataAugmenter( ...\r\n        'RandRotation',[-5 5],'RandXReflection',1,...\r\n        'RandYReflection',1,'RandXShear',[-0.05 0.05],'RandYShear',[-0.05 0.05]);\r\n    \r\n    % Resizing all training images to [224 224] for ResNet architecture<\/span>\r\n    auimds = augmentedImageDatastore([224 224],imdsTrain,'DataAugmentation',augmenter);\r\n    \r\n    % Training<\/span>\r\n    netTransfer = trainNetwork(auimds,lgraph,options);\r\n    \r\n    % Resizing all testing images to [224 224] for ResNet architecture   <\/span>\r\n    augtestimds = augmentedImageDatastore([224 224],imdsTest);\r\n   \r\n    % Testing and their corresponding Labels and Posterior for each Case<\/span>\r\n    [predicted_labels(test_idx),posterior(test_idx,:)] = classify(netTransfer,augtestimds);\r\n    \r\n    % Save the Independent ResNet Architectures obtained for each Fold<\/span>\r\n    save(sprintf('ResNet50_%d_among_%d_folds',fold_idx,num_folds),'netTransfer','test_idx','train_idx');\r\n    \r\n    % Clearing unnecessary variables <\/span>\r\n    clearvars -except fold_idx num_folds num_images predicted_labels posterior imds netTransfer;\r\n    \r\nend<\/pre>\r\n
<\/h6>\r\n
\"\"
Each fold will display a training plot while training, for brevity this is only showing the final training plot, fold 10 of 10.<\/p><\/div>\r\n
Performance Study<\/h2>\r\nLet's measure the performance of our algorithm in terms of confusion matrix - This metric also gives a good idea of the performance in terms of precision and recall. We believe overall accuracy is a good indicator as the testing dataset utilized in this study is uniformly distributed (in terms of images belonging to each category).\r\n
<\/h6>\r\nConfusion Matrix<\/strong>\r\n
<\/h6>\r\n
% Actual Labels<\/span>\r\nactual_labels=imds.Labels;\r\n\r\n% Confusion Matrix<\/span>\r\nfigure;\r\nplotconfusion(actual_labels,predicted_labels')\r\ntitle('Confusion Matrix: ResNet');\r\n<\/pre>\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\nROC Curve<\/strong>\r\n
<\/h6>\r\nROC would assist the doctors in choosing their operating point in terms of false positive and detection rate.\r\n
test_labels=double(nominal(imds.Labels));\r\n\r\n% ROC Curve - Our target class is the first class in this scenario <\/span>\r\n[fp_rate,tp_rate,T,AUC]=perfcurve(test_labels,posterior(:,1),1);\r\nfigure;\r\nplot(fp_rate,tp_rate,'b-');\r\ngrid on;\r\nxlabel('False Positive Rate');\r\nylabel('Detection Rate');<\/pre>\r\n\"\"\r\n
% Area under the ROC curve value<\/span>\r\nAUC<\/pre>\r\n\"\"\r\n
<\/h6>\r\n
Class Activation Mapping<\/h2>\r\n
<\/h6>\r\nWe visualize the Class Activation Mapping (CAM) results for these networks for different COVID-19 cases using the code: https:\/\/www.mathworks.com\/help\/deeplearning\/examples\/investigate-network-predictions-using-class-activation-mapping.html<\/a>.\r\n\r\nThis would help in providing insights behind the algorithm's decision to the doctors.\r\n
<\/h6>\r\nHere are the results obtained for various cases:\r\n
<\/h6>\r\n\r\n\r\n\r\n
\"\"<\/td>\r\n\"\"<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n
<\/h6>\r\n
Test on other publicly available datasets<\/h2>\r\n
<\/h6>\r\nTo study and analyze the performance of the algorithm futher, we determine the probability of COVID-19 for other publicly available datasets without any COVID-19 labels. We utilize the cases marked as either Normal, Bacterial Pneumonia or Viral Pneumonia by radiologists presented in [2]. Aforementioned, each network was trained on a different set of images from COVID-19 dataset. Any image with a covid posterior value greater than 0.5 is considered to be a False Positive (FP). Results clearly indicate that our algorithm's specificity and sensitivity is high. Time taken per test case is nearly 13 milliseconds on a single-core GPU.\r\n
<\/h6>\r\n\"\"\r\n
<\/h6>\r\n
<\/h6>\r\n
Conclusions<\/h2>\r\nIn this blog, we have presented a simple deep learning-based classification approach for CAD of COVID-19. Classification algorithm using ResNet performed relatively well with a high overall accuracy and AUC. Performance of transfer learning approaches clearly reiterates that CNN based classification models are good in extracting features. Algorithm can be easily re-trained with new sets of labeled images to enhance the performance further. Combining these results with other exisiting architectures would provide a boost in performance both in terms of AUC and overall accuracy. A comprehensive study of these algorithms both in terms of computation (memory and time) and performance provides the subject matter experts to choose algorithms based on their choice. CAD would be of great help for the doctors for COVID-19 screening and would help in providing a valuable second opinion.\r\n
<\/h6>\r\n
References<\/h2>\r\n[1] Narayanan, B. N., De Silva, M. S., Hardie, R. C., Kueterman, N. K., & Ali, R. (2019). \"Understanding Deep Neural Network Predictions for Medical Imaging Applications\". arXiv preprint arXiv:1912.09621.\r\n
<\/h6>\r\n[2] Narayanan, B. N., Davuluru, V. S. P., & Hardie, R. C. (2020, March). \"Two-stage deep learning architecture for pneumonia detection and its diagnosis in chest radiographs\". In Medical Imaging 2020: Imaging Informatics for Healthcare, Research, and Applications (Vol. 11318, p. 113180G). International Society for Optics and Photonics.\r\n
<\/h6>\r\n
Preprocessing Function<\/h2>\r\n
<\/h6>\r\n
function Iout = preprocess_Xray(filename)\r\n% This function preprocesses the given X-ray image by converting it into<\/span>\r\n% grayscale if required and later converting to 3-channel image to adapt to<\/span>\r\n% existing deep learning architectures <\/span>\r\n%<\/span>\r\n% Author: Barath Narayanan<\/span>\r\n% Date: 3\/17\/2020<\/span>\r\n\r\n% Read the Filename<\/span>\r\nI = imread(filename);\r\n\r\n% Some images might be RGB, convert them to Grayscale<\/span>\r\nif ~ismatrix(I)\r\n    I=rgb2gray(I); \r\nend\r\n\r\n% Replicate the image 3 times to create an RGB image<\/span>\r\nIout = cat(3,I,I,I);\r\n\r\nend<\/pre>\r\n
<\/h6>\r\nDo you have any questions for Barath on this post? Leave a comment below\r\n
<\/h6>","protected":false},"excerpt":{"rendered":"
<\/div>
\r\n\r\n\r\nI'm pleased to publish another post from Barath Narayanan, University of Dayton Research Institute (UDRI), LinkedIn Profile.\r\n\r\nCo-author: Dr. Russell C. Hardie, University of Dayton... read more >><\/a><\/p>","protected":false},"author":156,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9],"tags":[],"_links":{"self":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4043"}],"collection":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/users\/156"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/comments?post=4043"}],"version-history":[{"count":5,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4043\/revisions"}],"predecessor-version":[{"id":4225,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/posts\/4043\/revisions\/4225"}],"wp:attachment":[{"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/media?parent=4043"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/categories?post=4043"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.mathworks.com\/deep-learning\/wp-json\/wp\/v2\/tags?post=4043"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}