{"id":4935,"date":"2021-01-08T16:41:14","date_gmt":"2021-01-08T15:41:14","guid":{"rendered":"https:\/\/blogs.mathworks.com\/student-lounge\/?p=4935"},"modified":"2021-04-06T21:38:39","modified_gmt":"2021-04-06T19:38:39","slug":"matlab-benchmark-code-for-wids-datathon-2021","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/student-lounge\/2021\/01\/08\/matlab-benchmark-code-for-wids-datathon-2021\/","title":{"rendered":"MATLAB Benchmark Code for WiDS Datathon 2021"},"content":{"rendered":"

Introduction<\/h1>\n

Hello all, I am Neha Goel, Technical Lead for AI\/Data Science competitions on the MathWorks Student Competition team. MathWorks is excited to support WiDS Datathon 2021 by providing complimentary MATLAB Licenses, tutorials, and getting started resources to each participant.<\/p>\n

To request your complimentary license, go to the MathWorks site<\/a>, click the \u201cRequest Software\u201d button, and fill out the software request form.<\/strong> You will get your license within 72 business hours.<\/em><\/p>\n

The WiDS Datathon 2021 focuses on patient health through data from MIT\u2019s GOSSIS (Global Open Source Severity of Illness Score) initiative. Brought to you by the Global WiDS team, the West Big Data Innovation Hub<\/a>, and the WiDS Datathon Committee, open until March 1st, 2021.\u00a0Learn more about the\u00a0WiDS Datathon<\/a>, or\u00a0register to participate<\/a>\u00a0today.<\/p>\n

The Datathon task is to train a model that takes as input the patient record data and outputs a prediction of how likely it is that the patient have been diagnosed with a certain type of diabetes which could inform treatment in the ICU. In this blog post I will walk through basic starter code workflow in MATLAB. Additional resources for other training methods are linked at the bottom of the blog post.<\/p>\n

Load & Prepare Data<\/h1>\n

This dataset presents an opportunity to learn about the data modelling and processing challenges a real-world data problem brings. In this blog, I will talk about some basic methods to handle data challenges. To learn some more methods you can go through this Data Science Tutorial video series<\/a> .<\/p>\n

Step 1: Load Data<\/h2>\n

Register for the competition and download the data files from Kaggle. “TrainingWiDS2021.csv<\/em>” is the training data file and “UnlabeledWiDS2021.csv<\/em>” is the test data.
\nOnce you download the files, make sure that the files are in the MATLAB path. Here I use the readtable<\/a> function to read the files and store it as tables. TreatAsEmpty<\/em> is the placeholder text to treat empty values to numeric columns in file. Table elements corresponding to characters ‘NA’<\/em> will be set as ‘NaN’<\/em> when imported. You can also import data using the MATLAB Import tool<\/a>.<\/p>\n

TrainSet = readtable('TrainingWiDS2021.csv','TreatAsEmpty','NA');<\/pre>\n
Step 2: Clean Data<\/h2>\n
The biggest challenge with this dataset is that the data is messy. 180 predictor columns,\u00a0 130157 observations with lot of missing values. Data transformation and modelling will be the key area to work on to avoid overfitting the problem.
\nUsing the summary<\/em> function, I analyzed the types of the predictors, the min, max, median values and number of missing values for each predictor column. This helped me derive relevant assumptions to clean the data.<\/p>\n
summary(TrainSet);\r\n<\/pre>\n
\"\"<\/p>\n
There are many different approaches to work with the missing values and predictor selection. We will go through one of the basic approaches in this blog. You can also refer to this document to learn about other methods : Clean Messy and Missing Data<\/a>.<\/p>\n
Note: This approach of data cleaning demonstrated is chosen arbitrarily to cut down number of predictor columns.<\/em><\/p>\n
Remove the character columns of the table<\/strong><\/p>\n
The reason behind this is that the algorithm I chose to train the model is fitclinear<\/em> and it only allows numeric matrix as the input arguments. Hence removing variables of “Categorical” or “non-numeric” type.<\/p>\n
TrainSet = removevars(TrainSet, {'ethnicity','gender','hospital_admit_source','icu_admit_source',...\r\n'icu_stay_type','icu_type'});<\/pre>\n
Remove minimum values from all the vitals predictors<\/strong><\/p>\n
After analyzing the\u00a0 DataDictionaryWiDS2021.csv<\/em>\u00a0 \u00a0file provided with the Kaggle data, I noticed that the even columns from column 40 to 166 correspond to minimum values of predictors in the vital<\/em> category.<\/p>\n
TrainSet = removevars(TrainSet,(40:2:166));<\/pre>\n
Remove the observations which have 30 or more missing predictors<\/strong><\/p>\n
The other assumption I made is the observations (patients) which have 30 or more missing predictor values can be removed.<\/p>\n
TrainSet = rmmissing(TrainSet,1,'MinNumMissing',30);\r\n<\/pre>\n
Remove the observations which have 30 or more missing predictors<\/strong><\/p>\n
The other assumption I made is the observations (patients) which have 30 or more missing predictor values can be removed.<\/p>\n
TrainSet = rmmissing(TrainSet,1,'MinNumMissing',30);<\/pre>\n
Fill the missing values<\/strong><\/p>\n
The next step is to fill in all the NaN values. One approach is to use the fillmissing<\/a> function to fill data using linear interpolation. Other approaches include replacing NaN values with mean or median values and removing the outliers using the CurveFitting app<\/a>.<\/p>\n
TrainSet = convertvars(TrainSet,@iscategorical,'double');\r\nTrainSet = fillmissing(TrainSet,'linear');<\/pre>\n
In this step I move our label predictor diabetes_mellitus<\/em> to the last column of the table because for some algorithms in MATLAB and in Classification learner app the last column is the default response variable.<\/p>\n
TrainSet = movevars(TrainSet,'diabetes_mellitus','After',110);<\/pre>\n
Step 3: Create Training Data<\/h2>\n
Once I have the cleaned training data. I separate the label predictor diabetes_mellitus<\/em> from the training set and create two separate tables XTrain:<\/em> Predictor data , YTrain<\/em>: Class labels<\/p>\n
XTrain = removevars(TrainSet,{'diabetes_mellitus'});\r\nYTrain = TrainSet.diabetes_mellitus;<\/pre>\n
Step 4: Create Test Data<\/h2>\n
Download the UnlabeledWiDS2021.csv<\/em>\u00a0file from Kaggle. Read the file using the readtable<\/em> function to store it as a table. You can also use Import tool to load the data.<\/p>\n
XTest = readtable('UnlabeledWiDS2021.csv','TreatAsEmpty','NA');<\/pre>\n
I used a similar approach for cleaning test data as the training data above. XTest is the test data with no label predictor.<\/p>\n
Remove the character columns of the table<\/strong><\/p>\n
XTest = removevars(XTest, {'ethnicity','gender','hospital_admit_source',...\r\n'icu_admit_source','icu_stay_type','icu_type'});<\/pre>\n
Remove minimum values from all the vitals predictors<\/strong><\/p>\n
The minimum values of vital<\/em> category correspond to the even columns from column 40 to 166.<\/p>\n
XTest = removevars(XTest, (40:2:166));<\/pre>\n
Fill the missing values<\/strong><\/p>\n
XTest = convertvars(XTest,@iscategorical,'double');\r\nXTest = fillmissing(XTest,'linear');<\/pre>\n
Step 5: Train a Model<\/h2>\n
In MATLAB you can train a model using two different methods.<\/p>\n
    \n
  1. Using custom MATLAB machine learning algorithm functions<\/li>\n
  2. Training the model using Classification learner app.<\/li>\n<\/ol>\n
    Here I walkthrough steps for doing both the methods. I would encourage to try both the approaches and train the model using different algorithms and parameters. It will help in optimization and comparing different model’s scores.<\/p>\n
    Option 1: Using custom algorithms<\/h2>\n
    A Binary classification problem can be approached using various algorithms like Decision tress, svm and logistic regression. Here I train using fitclinear<\/a> classification model. It trains the linear binary classification models with high dimensional predictor data.<\/p>\n
    Convert the table to a numeric matrix because fitclinear<\/em> function takes only numeric matrix as an input argument.<\/p>\n
    XTrainMat = table2array(XTrain);\r\nXTestMat = table2array(XTest);<\/pre>\n
    Fit the model<\/strong><\/p>\n
    The name value pair input arguments within the function gives the options of tuning the model. Here I use solver as sparsa (Sparse Reconstruction by Separable Approximation), which has default lasso<\/em> regularization. To optimize the model, I do some Hyperparameter Optimization.<\/p>\n
    ‘OptimizeHyperparameters’<\/em> as ‘auto’<\/em> uses {lambda, learner}<\/em> and acquisition<\/em> function name lets you modify the behavior when the function is overexploiting an area per second.<\/p>\n
    You can further cross validate the data within input arguments using cross-validation options: crossval, KFold, CVPartition<\/em> etc. Check out the fitclinear<\/em><\/a> document to know about input arguments.<\/p>\n
    Mdl = fitclinear(XTrainMat,YTrain,'ObservationsIn','rows','Solver','sparsa',...\r\n'OptimizeHyperparameters','auto','HyperparameterOptimizationOptions',...\r\nstruct('AcquisitionFunctionName','expected-improve\r\nment-plus'));<\/pre>\n
     <\/p>\n
    \"\"\u00a0\"\"<\/p>\n
    Predict on the Test Set<\/strong><\/p>\n
    Once we have your model ready, you can perform predictions on your test set using predict function. It takes as input the fitted model and Test data with similar predictors as training data. The output is the predicted labels and scores.<\/p>\n
    [label,scores] = predict(Mdl,XTestMat);<\/pre>\n
    Option 2: Using Classification Learner App<\/h2>\n
    Second method of training the model is by using the Classification Learner app<\/a>. It lets you interactively train, validate and tune classification model. Let’s see the steps to work with it.<\/p>\n