{"id":10714,"date":"2024-01-01T06:00:09","date_gmt":"2024-01-01T11:00:09","guid":{"rendered":"https:\/\/blogs.mathworks.com\/student-lounge\/?p=10714"},"modified":"2024-01-31T14:06:38","modified_gmt":"2024-01-31T19:06:38","slug":"predicting-timely-diagnosis-of-metastatic-breast-cancer-for-the-wids-datathon-2024","status":"publish","type":"post","link":"https:\/\/blogs.mathworks.com\/student-lounge\/2024\/01\/01\/predicting-timely-diagnosis-of-metastatic-breast-cancer-for-the-wids-datathon-2024\/","title":{"rendered":"Predicting Timely Diagnosis of Metastatic Breast Cancer for the WiDS Datathon 2024"},"content":{"rendered":"
\n
In today\u2019s blog, Grace Woolson <\/a>will show how you can use MATLAB and machine learning to make meaningful deductions from healthcare data for patients who have been diagnosed with metastatic breast cancer. Over to you Grace!<\/div>\n

Introduction<\/h2>\n
In this blog, I will show how you can use MATLAB for the WiDS Datathon 2024<\/a> using the dataset for the WiDS Datathon #1, which runs from January 9th 2024 – March 1st 2024. This challenge tasks participants with creating a model that can predict whether or not a patient with metastatic breast cancer will receive a diagnosis within 90 days based on patient and environmental data. This can help identify relationships between demographics or environmental hazards with the likelihood of getting timely treatment. Please note that this blog is based on a subset of the data and there may be slight differences between this dataset and the one provided by WiDS.<\/div>\n
MathWorks is happy to support participants of the Women in Data Science Datathon 2024 by providing complimentary MATLAB licenses, tutorials, workshops, and additional resources. To request complimentary licenses for you and your teammates, go to this <\/span>MathWorks site<\/span><\/a>, click the \u201cRequest Software\u201d button, and fill out the software request form.<\/span><\/div>\n
This tutorial will walk through the following steps of the model-making process:<\/div>\n
    \n
  1. Importing a Tabular Dataset<\/a><\/li>\n
  2. Preprocessing the Data<\/a><\/li>\n
  3. Exploring and Analyzing Tabular Data<\/a><\/li>\n
  4. Choosing and Creating Features<\/a><\/li>\n
  5. Training a Machine Learning Model<\/a><\/li>\n
  6. Evaluating a Machine Learning Model<\/a><\/li>\n
  7. Making New Predictions and Exporting Submissions<\/a><\/li>\n<\/ol>\n

    Import Data<\/h2>\n
    First, make sure the ‘Current Folder’ is the folder where you saved the data. If you have not already done so, you can download the data from Kaggle after you register<\/a> for the datathon. The data is provided as a .CSV file, so we can use the readtable<\/span><\/a> function to import the whole file as a table.<\/div>\n
    \n
    \n
    dataFolder = fullfile(pwd);<\/span><\/div>\n<\/div>\n
    \n
    trainDataFilename = ‘Training.csv’<\/span>;<\/span><\/div>\n<\/div>\n
    \n
    allTrainData = readtable(fullfile(dataFolder, trainDataFilename))<\/span><\/div>\n
    \n
    \n
    \n
    allTrainData = 12906\u00d783 table <\/span><\/div>\n
    \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
    <\/th>\npatient_id<\/th>\npatient_race<\/th>\npayer_type<\/th>\npatient_state<\/th>\npatient_zip3<\/th>\npatient_age<\/th>\npatient_gender<\/th>\nbmi<\/th>\nbreast_cancer_diagnosis_code<\/th>\nbreast_cancer_diagnosis_desc<\/th>\nmetastatic_cancer_diagnosis_code<\/th>\nmetastatic_first_novel_treatment<\/th>\nmetastatic_first_novel_treatment_type<\/th>\nRegion<\/th>\nDivision<\/th>\npopulation<\/th>\ndensity<\/th>\nage_median<\/th>\nage_under_10<\/th>\nage_10_to_19<\/th>\nage_20s<\/th>\nage_30s<\/th>\nage_40s<\/th>\nage_50s<\/th>\nage_60s<\/th>\nage_70s<\/th>\nage_over_80<\/th>\nmale<\/th>\nfemale<\/th>\nmarried<\/th>\n<\/tr>\n<\/thead>\n
    1<\/th>\n475714<\/td>\n”<\/td>\n‘MEDICAID’<\/td>\n‘CA’<\/td>\n924<\/td>\n84<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50919’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0unspecified\u00a0female\u00a0breast’<\/td>\n‘C7989’<\/td>\n”<\/td>\n”<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.1438e+04<\/td>\n1.1896e+03<\/td>\n30.6429<\/td>\n16.0143<\/td>\n15.5429<\/td>\n17.6143<\/td>\n14.0143<\/td>\n11.6143<\/td>\n11.5571<\/td>\n7.5714<\/td>\n4<\/td>\n2.1000<\/td>\n49.8571<\/td>\n50.1429<\/td>\n36.5714<\/td>\n<\/tr>\n
    2<\/th>\n349367<\/td>\n‘White’<\/td>\n‘COMMERCIAL’<\/td>\n‘CA’<\/td>\n928<\/td>\n62<\/td>\n‘F’<\/td>\n28.4900<\/td>\n‘C50411’<\/td>\n‘Malig\u00a0neoplm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0right\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\n”<\/td>\n”<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.9122e+04<\/td>\n2.2959e+03<\/td>\n38.2000<\/td>\n11.8788<\/td>\n13.3545<\/td>\n14.2303<\/td>\n13.4182<\/td>\n13.3333<\/td>\n14.0606<\/td>\n10.2485<\/td>\n5.9515<\/td>\n3.5030<\/td>\n49.8939<\/td>\n50.1061<\/td>\n50.2455<\/td>\n<\/tr>\n
    3<\/th>\n138632<\/td>\n‘White’<\/td>\n‘COMMERCIAL’<\/td>\n‘TX’<\/td>\n760<\/td>\n43<\/td>\n‘F’<\/td>\n38.09<\/td>\n‘C50112’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0central\u00a0portion\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\n”<\/td>\n”<\/td>\n‘South’<\/td>\n‘West\u00a0South\u00a0Central’<\/td>\n2.1997e+04<\/td>\n626.2367<\/td>\n37.9067<\/td>\n13.0283<\/td>\n14.4633<\/td>\n12.5317<\/td>\n13.5450<\/td>\n12.8600<\/td>\n12.7700<\/td>\n11.4267<\/td>\n6.5650<\/td>\n2.8117<\/td>\n50.1233<\/td>\n49.8767<\/td>\n55.7533<\/td>\n<\/tr>\n
    4<\/th>\n617843<\/td>\n‘White’<\/td>\n‘COMMERCIAL’<\/td>\n‘CA’<\/td>\n926<\/td>\n45<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50212’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0upper-inner\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\n”<\/td>\n”<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.2795e+04<\/td>\n1.8962e+03<\/td>\n42.8714<\/td>\n10.0714<\/td>\n12.1357<\/td>\n12.5381<\/td>\n12.4643<\/td>\n12.6500<\/td>\n14.8476<\/td>\n12.2810<\/td>\n8.2167<\/td>\n4.7595<\/td>\n49.0667<\/td>\n50.9333<\/td>\n52.6048<\/td>\n<\/tr>\n
    5<\/th>\n817482<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘ID’<\/td>\n836<\/td>\n55<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘1749’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified’<\/td>\n‘C773’<\/td>\n”<\/td>\n”<\/td>\n‘West’<\/td>\n‘Mountain’<\/td>\n1.0886e+04<\/td>\n116.8860<\/td>\n43.4735<\/td>\n10.8240<\/td>\n13.9760<\/td>\n9.4920<\/td>\n10.3640<\/td>\n12.6000<\/td>\n14.9920<\/td>\n14.8360<\/td>\n9.4620<\/td>\n3.4660<\/td>\n52.3120<\/td>\n47.6880<\/td>\n57.8820<\/td>\n<\/tr>\n
    6<\/th>\n111545<\/td>\n‘White’<\/td>\n‘MEDICARE\u00a0ADVANTAGE’<\/td>\n‘NY’<\/td>\n141<\/td>\n66<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘1749’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified’<\/td>\n‘C7981’<\/td>\n”<\/td>\n”<\/td>\n‘Northeast’<\/td>\n‘Middle\u00a0Atlantic’<\/td>\n5.6438e+03<\/td>\n219.3629<\/td>\n45.1800<\/td>\n8.5114<\/td>\n14.8571<\/td>\n11.0886<\/td>\n9.7543<\/td>\n13.6143<\/td>\n13.3743<\/td>\n15.6857<\/td>\n9.4457<\/td>\n3.6457<\/td>\n50.9114<\/td>\n49.0914<\/td>\n51.3229<\/td>\n<\/tr>\n
    7<\/th>\n914071<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘CA’<\/td>\n900<\/td>\n51<\/td>\n‘F’<\/td>\n29.0500<\/td>\n‘C50912’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C779’<\/td>\n”<\/td>\n”<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.6054e+04<\/td>\n5.2943e+03<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n<\/tr>\n
    8<\/th>\n479368<\/td>\n‘White’<\/td>\n‘COMMERCIAL’<\/td>\n‘IL’<\/td>\n619<\/td>\n60<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50512’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0lower-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\n”<\/td>\n”<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n3.4041e+03<\/td>\n25.7333<\/td>\n42.7900<\/td>\n11.9833<\/td>\n13.2567<\/td>\n9.5733<\/td>\n12.4000<\/td>\n11.8133<\/td>\n13.5767<\/td>\n14.0433<\/td>\n8.5267<\/td>\n4.8533<\/td>\n49.2833<\/td>\n50.7167<\/td>\n55.8867<\/td>\n<\/tr>\n
    9<\/th>\n994014<\/td>\n‘White’<\/td>\n‘MEDICARE\u00a0ADVANTAGE’<\/td>\n”<\/td>\n973<\/td>\n82<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘1744’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0female\u00a0breast’<\/td>\n‘C7800’<\/td>\n”<\/td>\n”<\/td>\n”<\/td>\n”<\/td>\n1.0111e+04<\/td>\n240.5785<\/td>\n44.9159<\/td>\n9.0646<\/td>\n12.1000<\/td>\n11.1385<\/td>\n11.5123<\/td>\n10.5354<\/td>\n12.7292<\/td>\n18.5462<\/td>\n10.7431<\/td>\n3.6400<\/td>\n52.5846<\/td>\n47.4154<\/td>\n52.9938<\/td>\n<\/tr>\n
    10<\/th>\n155485<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘IL’<\/td>\n617<\/td>\n64<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50912’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\n”<\/td>\n”<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n4.4353e+03<\/td>\n68.0019<\/td>\n41.3000<\/td>\n12.8358<\/td>\n13.6811<\/td>\n10.5245<\/td>\n11.9377<\/td>\n11.6585<\/td>\n13.5774<\/td>\n13.7434<\/td>\n7.6868<\/td>\n4.3415<\/td>\n49.3962<\/td>\n50.6038<\/td>\n57.8962<\/td>\n<\/tr>\n
    11<\/th>\n875977<\/td>\n”<\/td>\n‘MEDICARE\u00a0ADVANTAGE’<\/td>\n‘MI’<\/td>\n488<\/td>\n67<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50412’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C799’<\/td>\n”<\/td>\n”<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n8101<\/td>\n246.2810<\/td>\n40.2782<\/td>\n11.0456<\/td>\n14.7684<\/td>\n13.3848<\/td>\n11.4671<\/td>\n11.2203<\/td>\n14.8975<\/td>\n12.5899<\/td>\n7.1494<\/td>\n3.4709<\/td>\n51.3228<\/td>\n48.6772<\/td>\n49.0658<\/td>\n<\/tr>\n
    12<\/th>\n343914<\/td>\n”<\/td>\n‘MEDICARE\u00a0ADVANTAGE’<\/td>\n‘CA’<\/td>\n900<\/td>\n66<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘1749’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified’<\/td>\n‘C7800’<\/td>\n”<\/td>\n”<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.6054e+04<\/td>\n5.2943e+03<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n<\/tr>\n
    13<\/th>\n266700<\/td>\n‘White’<\/td>\n‘COMMERCIAL’<\/td>\n‘MI’<\/td>\n480<\/td>\n58<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50812’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0ovrlp\u00a0sites\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C781’<\/td>\n”<\/td>\n”<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n1.6938e+04<\/td>\n894.1681<\/td>\n42.9348<\/td>\n10.5116<\/td>\n11.8130<\/td>\n11.9217<\/td>\n12.4043<\/td>\n12.4304<\/td>\n15.2710<\/td>\n13.8826<\/td>\n7.8522<\/td>\n3.8971<\/td>\n50.0217<\/td>\n49.9783<\/td>\n50.9565<\/td>\n<\/tr>\n
    14<\/th>\n437659<\/td>\n”<\/td>\n”<\/td>\n‘IL’<\/td>\n606<\/td>\n82<\/td>\n‘F’<\/td>\nundefined<\/td>\n‘1749’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified’<\/td>\n‘C779’<\/td>\n”<\/td>\n”<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n4.8671e+04<\/td>\n6.4314e+03<\/td>\n35.7554<\/td>\n10.4286<\/td>\n10.6518<\/td>\n18.3107<\/td>\n18.9036<\/td>\n11.9696<\/td>\n11.7268<\/td>\n9.6839<\/td>\n5.4071<\/td>\n2.8911<\/td>\n48.6964<\/td>\n51.3036<\/td>\n35.9304<\/td>\n<\/tr>\n
    \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
    <\/div>\n
    <\/div>\n
    <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
    I want to see some high-level statistics about the data, so I’ll use the summary<\/span><\/a> function to get an idea of what kind of information we have.<\/div>\n
    \n
    \n
    summary(allTrainData)<\/span><\/div>\n
    \n
    \n
    \n

    Variables:<\/p>\n

    patient_id<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.0006e+05
    \nMedian 5.4352e+05
    \nMax 9.999e+05<\/p>\n

    patient_race<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    payer_type<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    patient_state<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    patient_zip3<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 101
    \nMedian 554
    \nMax 999<\/p>\n

    patient_age<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 18
    \nMedian 59
    \nMax 91<\/p>\n

    patient_gender<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    bmi<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 14
    \nMedian 28.19
    \nMax 85
    \nNumMissing 8965<\/p>\n

    breast_cancer_diagnosis_code<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    breast_cancer_diagnosis_desc<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    metastatic_cancer_diagnosis_code<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    metastatic_first_novel_treatment<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    metastatic_first_novel_treatment_type<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    Region<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    Division<\/strong>: 12906\u00d71 cell array of character vectors<\/p>\n

    population<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 635.55
    \nMedian 19154
    \nMax 71374
    \nNumMissing 1<\/p>\n

    density<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.91667
    \nMedian 700.34
    \nMax 21172
    \nNumMissing 1<\/p>\n

    age_median<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 20.6
    \nMedian 40.639
    \nMax 54.57
    \nNumMissing 1<\/p>\n

    age_under_10<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 11.039
    \nMax 17.675
    \nNumMissing 1<\/p>\n

    age_10_to_19<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 6.3143
    \nMedian 12.924
    \nMax 35.3
    \nNumMissing 1<\/p>\n

    age_20s<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 5.925
    \nMedian 12.538
    \nMax 62.1
    \nNumMissing 1<\/p>\n

    age_30s<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.5
    \nMedian 12.443
    \nMax 25.471
    \nNumMissing 1<\/p>\n

    age_40s<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.8
    \nMedian 12.124
    \nMax 17.82
    \nNumMissing 1<\/p>\n

    age_50s<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 13.568
    \nMax 21.661
    \nNumMissing 1<\/p>\n

    age_60s<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.2
    \nMedian 12.533
    \nMax 29.855
    \nNumMissing 1<\/p>\n

    age_70s<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 7.3169
    \nMax 19
    \nNumMissing 1<\/p>\n

    age_over_80<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 3.8
    \nMax 18.825
    \nNumMissing 1<\/p>\n

    male<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 39.725
    \nMedian 49.976
    \nMax 61.6
    \nNumMissing 1<\/p>\n

    female<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 38.4
    \nMedian 50.024
    \nMax 60.275
    \nNumMissing 1<\/p>\n

    married<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.9
    \nMedian 49.434
    \nMax 66.903
    \nNumMissing 1<\/p>\n

    divorced<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.2
    \nMedian 12.653
    \nMax 19.831
    \nNumMissing 1<\/p>\n

    never_married<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 13.44
    \nMedian 32.004
    \nMax 98.9
    \nNumMissing 1<\/p>\n

    widowed<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 5.5208
    \nMax 23.055
    \nNumMissing 1<\/p>\n

    family_size<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.5504
    \nMedian 3.1665
    \nMax 4.1723
    \nNumMissing 4<\/p>\n

    family_dual_income<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 19.312
    \nMedian 52.592
    \nMax 70.925
    \nNumMissing 4<\/p>\n

    income_household_median<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 29222
    \nMedian 69803
    \nMax 1.6412e+05
    \nNumMissing 4<\/p>\n

    income_household_under_5<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.75
    \nMedian 2.8382
    \nMax 19.62
    \nNumMissing 4<\/p>\n

    income_household_5_to_10<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.36154
    \nMedian 2.1604
    \nMax 11.872
    \nNumMissing 4<\/p>\n

    income_household_10_to_15<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.0154
    \nMedian 3.7171
    \nMax 14.278
    \nNumMissing 4<\/p>\n

    income_household_15_to_20<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.0278
    \nMedian 3.7712
    \nMax 12.918
    \nNumMissing 4<\/p>\n

    income_household_20_to_25<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.1
    \nMedian 4.0421
    \nMax 14.35
    \nNumMissing 4<\/p>\n

    income_household_25_to_35<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.65
    \nMedian 8.4353
    \nMax 18.34
    \nNumMissing 4<\/p>\n

    income_household_35_to_50<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.7
    \nMedian 11.793
    \nMax 24.075
    \nNumMissing 4<\/p>\n

    income_household_50_to_75<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 4.95
    \nMedian 17.076
    \nMax 27.13
    \nNumMissing 4<\/p>\n

    income_household_75_to_100<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 4.7333
    \nMedian 12.677
    \nMax 24.8
    \nNumMissing 4<\/p>\n

    income_household_100_to_150<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 4.2889
    \nMedian 16.016
    \nMax 31.325
    \nNumMissing 4<\/p>\n

    income_household_150_over<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.84
    \nMedian 14.703
    \nMax 52.824
    \nNumMissing 4<\/p>\n

    income_household_six_figure<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 5.6926
    \nMedian 30.575
    \nMax 69.032
    \nNumMissing 4<\/p>\n

    income_individual_median<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 4316
    \nMedian 35253
    \nMax 88910
    \nNumMissing 1<\/p>\n

    home_ownership<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 15.85
    \nMedian 69.669
    \nMax 90.367
    \nNumMissing 4<\/p>\n

    housing_units<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 6994.4
    \nMax 25923
    \nNumMissing 1<\/p>\n

    home_value<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 60629
    \nMedian 2.4784e+05
    \nMax 1.8531e+06
    \nNumMissing 4<\/p>\n

    rent_median<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 448.4
    \nMedian 1168
    \nMax 2965.2
    \nNumMissing 4<\/p>\n

    rent_burden<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 17.416
    \nMedian 30.986
    \nMax 78.94
    \nNumMissing 4<\/p>\n

    education_less_highschool<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 10.843
    \nMax 34.325
    \nNumMissing 1<\/p>\n

    education_highschool<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 27.406
    \nMax 53.96
    \nNumMissing 1<\/p>\n

    education_some_college<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 7.2
    \nMedian 29.286
    \nMax 50.133
    \nNumMissing 1<\/p>\n

    education_bachelors<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.4657
    \nMedian 19.047
    \nMax 41.7
    \nNumMissing 1<\/p>\n

    education_graduate<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.0941
    \nMedian 10.796
    \nMax 51.84
    \nNumMissing 1<\/p>\n

    education_college_or_above<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 7.0488
    \nMedian 30.141
    \nMax 77.817
    \nNumMissing 1<\/p>\n

    education_stem_degree<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 23.915
    \nMedian 43.066
    \nMax 73
    \nNumMissing 1<\/p>\n

    labor_force_participation<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 30.7
    \nMedian 62.778
    \nMax 78.67
    \nNumMissing 1<\/p>\n

    unemployment_rate<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.82308
    \nMedian 5.4741
    \nMax 18.8
    \nNumMissing 1<\/p>\n

    self_employed<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.263
    \nMedian 12.748
    \nMax 25.538
    \nNumMissing 4<\/p>\n

    farmer<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 0.45493
    \nMax 26.729
    \nNumMissing 4<\/p>\n

    race_white<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 14.496
    \nMedian 70.878
    \nMax 98.444
    \nNumMissing 1<\/p>\n

    race_black<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.060976
    \nMedian 6.4103
    \nMax 69.66
    \nNumMissing 1<\/p>\n

    race_asian<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 2.9667
    \nMax 49.85
    \nNumMissing 1<\/p>\n

    race_native<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 0.43095
    \nMax 76.935
    \nNumMissing 1<\/p>\n

    race_pacific<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 0.054054
    \nMax 14.758
    \nNumMissing 1<\/p>\n

    race_other<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.0025641
    \nMedian 3.5136
    \nMax 33.189
    \nNumMissing 1<\/p>\n

    race_multiple<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.43333
    \nMedian 5.802
    \nMax 26.43
    \nNumMissing 1<\/p>\n

    hispanic<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0.19444
    \nMedian 11.983
    \nMax 91.005
    \nNumMissing 1<\/p>\n

    disabled<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 4.6
    \nMedian 12.884
    \nMax 35.156
    \nNumMissing 1<\/p>\n

    poverty<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 3.4333
    \nMedian 12.178
    \nMax 38.348
    \nNumMissing 4<\/p>\n

    limited_english<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 2.7472
    \nMax 26.755
    \nNumMissing 4<\/p>\n

    commute_time<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 12.461
    \nMedian 27.788
    \nMax 48.02
    \nNumMissing 1<\/p>\n

    health_uninsured<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.44
    \nMedian 7.4657
    \nMax 27.566
    \nNumMissing 1<\/p>\n

    veteran<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 1.2
    \nMedian 6.8471
    \nMax 25.2
    \nNumMissing 1<\/p>\n

    Ozone<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 30.939
    \nMedian 39.108
    \nMax 52.237
    \nNumMissing 29<\/p>\n

    PM25<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.636
    \nMedian 7.6866
    \nMax 11.169
    \nNumMissing 29<\/p>\n

    N02<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 2.7604
    \nMedian 15.589
    \nMax 31.505
    \nNumMissing 29<\/p>\n

    DiagPeriodL90D<\/strong>: 12906\u00d71 double<\/p>\n

    Values:<\/p>\n

    Min 0
    \nMedian 1
    \nMax 1<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

    Take some time to scroll through this summary and see what information or patterns you can learn! Here are some things I notice:<\/div>\n
      \n
    1. There are a lot of rows or variables<\/span> that just say “cell array of character vectors”, which doesn’t tell us much about the data.<\/li>\n
    2. There are a few variables that have a high ‘NumMissing’ value.<\/li>\n
    3. The numeric variables can have dramatically different minimums and maximums.<\/li>\n<\/ol>\n
      We can use these observations to make decisions about how we want to explore and preprocess the dataset.<\/div>\n

      Process and Clean the Data<\/h2>\n

      1. Convert text data to categorical<\/h3>\n
      Text data can be hard for machine learning algorithms to understand, so let’s go through and change each “cell array of character vectors” to a categorical<\/span><\/a>. This will help the algorithm sort the text into different categories instead of understanding it as a series of individual letters.<\/div>\n
      \n
      \n
      varTypes = varfun(@class, allTrainData, OutputFormat=“cell”<\/span>);<\/span><\/div>\n<\/div>\n
      \n
      catIdx = strcmp(varTypes, “cell”<\/span>);<\/span><\/div>\n<\/div>\n
      \n
      varNames = allTrainData.Properties.VariableNames;<\/span><\/div>\n<\/div>\n
      \n
      catVarNames = varNames(catIdx);<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      for <\/span>catNameIdx = 1:length(catVarNames)<\/span><\/div>\n<\/div>\n
      \n
      allTrainData.(catVarNames{catNameIdx}) = categorical(allTrainData.(catVarNames{catNameIdx}));<\/span><\/div>\n<\/div>\n
      \n
      end<\/span><\/span><\/div>\n<\/div>\n<\/div>\n

      2. Handle Missing Data<\/h3>\n
      Now I want to handle all that missing data I noticed earlier. I’ll go through each variable and specifically look at variables that are missing data for over half of the rows or observations<\/span>.<\/div>\n
      \n
      \n
      dataSum = summary(allTrainData);<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      for <\/span>nameIdx = 1:length(varNames)<\/span><\/div>\n<\/div>\n
      \n
      varName = varNames{nameIdx};<\/span><\/div>\n<\/div>\n
      \n
      varNumMissing = dataSum.(varName).NumMissing;<\/span><\/div>\n<\/div>\n
      \n
      if <\/span>varNumMissing > (height(allTrainData) \/ 2)<\/span><\/div>\n<\/div>\n
      \n
      disp(varName);<\/span><\/div>\n<\/div>\n
      \n
      disp(varNumMissing);<\/span><\/div>\n<\/div>\n
      \n
      end<\/span><\/span><\/div>\n<\/div>\n
      \n
      end<\/span><\/span><\/div>\n
      \n
      \n
      bmi<\/div>\n<\/div>\n
      \n
      8965<\/div>\n<\/div>\n
      \n
      metastatic_first_novel_treatment<\/div>\n<\/div>\n
      \n
      12882<\/div>\n<\/div>\n
      \n
      metastatic_first_novel_treatment_type<\/div>\n<\/div>\n
      \n
      12882<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      Let’s remove those variables entirely, since they might not be too helpful for our algorithm.<\/div>\n
      \n
      \n
      allTrainData = removevars(allTrainData, [“bmi”<\/span>, “metastatic_first_novel_treatment”<\/span>, “metastatic_first_novel_treatment_type”<\/span>])<\/span><\/div>\n
      \n
      \n
      \n
      allTrainData = 12906\u00d780 table <\/span><\/div>\n
      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
      <\/th>\npatient_id<\/th>\npatient_race<\/th>\npayer_type<\/th>\npatient_state<\/th>\npatient_zip3<\/th>\npatient_age<\/th>\npatient_gender<\/th>\nbreast_cancer_diagnosis_code<\/th>\nbreast_cancer_diagnosis_desc<\/th>\nmetastatic_cancer_diagnosis_code<\/th>\nRegion<\/th>\nDivision<\/th>\npopulation<\/th>\ndensity<\/th>\nage_median<\/th>\nage_under_10<\/th>\nage_10_to_19<\/th>\nage_20s<\/th>\nage_30s<\/th>\nage_40s<\/th>\nage_50s<\/th>\nage_60s<\/th>\nage_70s<\/th>\nage_over_80<\/th>\nmale<\/th>\nfemale<\/th>\nmarried<\/th>\ndivorced<\/th>\nnever_married<\/th>\nwidowed<\/th>\n<\/tr>\n<\/thead>\n
      1<\/th>\n475714<\/td>\n<undefined><\/td>\nMEDICAID<\/td>\nCA<\/td>\n924<\/td>\n84<\/td>\nF<\/td>\nC50919<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0unspecified\u00a0female\u00a0breast<\/td>\nC7989<\/td>\nWest<\/td>\nPacific<\/td>\n3.1438e+04<\/td>\n1.1896e+03<\/td>\n30.6429<\/td>\n16.0143<\/td>\n15.5429<\/td>\n17.6143<\/td>\n14.0143<\/td>\n11.6143<\/td>\n11.5571<\/td>\n7.5714<\/td>\n4<\/td>\n2.1000<\/td>\n49.8571<\/td>\n50.1429<\/td>\n36.5714<\/td>\n11.8857<\/td>\n47.1143<\/td>\n4.4429<\/td>\n<\/tr>\n
      2<\/th>\n349367<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n928<\/td>\n62<\/td>\nF<\/td>\nC50411<\/td>\nMalig\u00a0neoplm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nPacific<\/td>\n3.9122e+04<\/td>\n2.2959e+03<\/td>\n38.2000<\/td>\n11.8788<\/td>\n13.3545<\/td>\n14.2303<\/td>\n13.4182<\/td>\n13.3333<\/td>\n14.0606<\/td>\n10.2485<\/td>\n5.9515<\/td>\n3.5030<\/td>\n49.8939<\/td>\n50.1061<\/td>\n50.2455<\/td>\n9.8273<\/td>\n35.2909<\/td>\n4.6515<\/td>\n<\/tr>\n
      3<\/th>\n138632<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nTX<\/td>\n760<\/td>\n43<\/td>\nF<\/td>\nC50112<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0central\u00a0portion\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nSouth<\/td>\nWest\u00a0South\u00a0Central<\/td>\n2.1997e+04<\/td>\n626.2367<\/td>\n37.9067<\/td>\n13.0283<\/td>\n14.4633<\/td>\n12.5317<\/td>\n13.5450<\/td>\n12.8600<\/td>\n12.7700<\/td>\n11.4267<\/td>\n6.5650<\/td>\n2.8117<\/td>\n50.1233<\/td>\n49.8767<\/td>\n55.7533<\/td>\n12.3300<\/td>\n27.1950<\/td>\n4.7100<\/td>\n<\/tr>\n
      4<\/th>\n617843<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n926<\/td>\n45<\/td>\nF<\/td>\nC50212<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0upper-inner\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nPacific<\/td>\n3.2795e+04<\/td>\n1.8962e+03<\/td>\n42.8714<\/td>\n10.0714<\/td>\n12.1357<\/td>\n12.5381<\/td>\n12.4643<\/td>\n12.6500<\/td>\n14.8476<\/td>\n12.2810<\/td>\n8.2167<\/td>\n4.7595<\/td>\n49.0667<\/td>\n50.9333<\/td>\n52.6048<\/td>\n11.6238<\/td>\n31.1429<\/td>\n4.6238<\/td>\n<\/tr>\n
      5<\/th>\n817482<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nID<\/td>\n836<\/td>\n55<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC773<\/td>\nWest<\/td>\nMountain<\/td>\n1.0886e+04<\/td>\n116.8860<\/td>\n43.4735<\/td>\n10.8240<\/td>\n13.9760<\/td>\n9.4920<\/td>\n10.3640<\/td>\n12.6000<\/td>\n14.9920<\/td>\n14.8360<\/td>\n9.4620<\/td>\n3.4660<\/td>\n52.3120<\/td>\n47.6880<\/td>\n57.8820<\/td>\n14.9640<\/td>\n21.7600<\/td>\n5.4060<\/td>\n<\/tr>\n
      6<\/th>\n111545<\/td>\nWhite<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nNY<\/td>\n141<\/td>\n66<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC7981<\/td>\nNortheast<\/td>\nMiddle\u00a0Atlantic<\/td>\n5.6438e+03<\/td>\n219.3629<\/td>\n45.1800<\/td>\n8.5114<\/td>\n14.8571<\/td>\n11.0886<\/td>\n9.7543<\/td>\n13.6143<\/td>\n13.3743<\/td>\n15.6857<\/td>\n9.4457<\/td>\n3.6457<\/td>\n50.9114<\/td>\n49.0914<\/td>\n51.3229<\/td>\n11.7600<\/td>\n30.8314<\/td>\n6.0914<\/td>\n<\/tr>\n
      7<\/th>\n914071<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n900<\/td>\n51<\/td>\nF<\/td>\nC50912<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC779<\/td>\nWest<\/td>\nPacific<\/td>\n3.6054e+04<\/td>\n5.2943e+03<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n11.3015<\/td>\n50.4569<\/td>\n4.7662<\/td>\n<\/tr>\n
      8<\/th>\n479368<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nIL<\/td>\n619<\/td>\n60<\/td>\nF<\/td>\nC50512<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0lower-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n3.4041e+03<\/td>\n25.7333<\/td>\n42.7900<\/td>\n11.9833<\/td>\n13.2567<\/td>\n9.5733<\/td>\n12.4000<\/td>\n11.8133<\/td>\n13.5767<\/td>\n14.0433<\/td>\n8.5267<\/td>\n4.8533<\/td>\n49.2833<\/td>\n50.7167<\/td>\n55.8867<\/td>\n12.6400<\/td>\n24.5267<\/td>\n6.9433<\/td>\n<\/tr>\n
      9<\/th>\n994014<\/td>\nWhite<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\n<undefined><\/td>\n973<\/td>\n82<\/td>\nF<\/td>\n1744<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0female\u00a0breast<\/td>\nC7800<\/td>\n<undefined><\/td>\n<undefined><\/td>\n1.0111e+04<\/td>\n240.5785<\/td>\n44.9159<\/td>\n9.0646<\/td>\n12.1000<\/td>\n11.1385<\/td>\n11.5123<\/td>\n10.5354<\/td>\n12.7292<\/td>\n18.5462<\/td>\n10.7431<\/td>\n3.6400<\/td>\n52.5846<\/td>\n47.4154<\/td>\n52.9938<\/td>\n13.9323<\/td>\n27.5262<\/td>\n5.5615<\/td>\n<\/tr>\n
      10<\/th>\n155485<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nIL<\/td>\n617<\/td>\n64<\/td>\nF<\/td>\nC50912<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n4.4353e+03<\/td>\n68.0019<\/td>\n41.3000<\/td>\n12.8358<\/td>\n13.6811<\/td>\n10.5245<\/td>\n11.9377<\/td>\n11.6585<\/td>\n13.5774<\/td>\n13.7434<\/td>\n7.6868<\/td>\n4.3415<\/td>\n49.3962<\/td>\n50.6038<\/td>\n57.8962<\/td>\n10.8981<\/td>\n24.9547<\/td>\n6.2472<\/td>\n<\/tr>\n
      11<\/th>\n875977<\/td>\n<undefined><\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nMI<\/td>\n488<\/td>\n67<\/td>\nF<\/td>\nC50412<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC799<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n8101<\/td>\n246.2810<\/td>\n40.2782<\/td>\n11.0456<\/td>\n14.7684<\/td>\n13.3848<\/td>\n11.4671<\/td>\n11.2203<\/td>\n14.8975<\/td>\n12.5899<\/td>\n7.1494<\/td>\n3.4709<\/td>\n51.3228<\/td>\n48.6772<\/td>\n49.0658<\/td>\n13.6051<\/td>\n31.8848<\/td>\n5.4392<\/td>\n<\/tr>\n
      12<\/th>\n343914<\/td>\n<undefined><\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nCA<\/td>\n900<\/td>\n66<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC7800<\/td>\nWest<\/td>\nPacific<\/td>\n3.6054e+04<\/td>\n5.2943e+04<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n11.3015<\/td>\n50.4569<\/td>\n4.7662<\/td>\n<\/tr>\n
      13<\/th>\n266700<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nMI<\/td>\n480<\/td>\n58<\/td>\nF<\/td>\nC50812<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0ovrlp\u00a0sites\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC781<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n1.6938e+04<\/td>\n894.1681<\/td>\n42.9348<\/td>\n10.5116<\/td>\n11.8130<\/td>\n11.9217<\/td>\n12.4043<\/td>\n12.4304<\/td>\n15.2710<\/td>\n13.8826<\/td>\n7.8522<\/td>\n3.8971<\/td>\n50.0217<\/td>\n49.9783<\/td>\n50.9565<\/td>\n12.3145<\/td>\n30.8333<\/td>\n5.9014<\/td>\n<\/tr>\n
      14<\/th>\n437659<\/td>\n<undefined><\/td>\n<undefined><\/td>\nIL<\/td>\n606<\/td>\n82<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC779<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n4.8671e+04<\/td>\n6.4314e+03<\/td>\n35.7554<\/td>\n10.4286<\/td>\n10.6518<\/td>\n18.3107<\/td>\n18.9036<\/td>\n11.9696<\/td>\n11.7268<\/td>\n9.6839<\/td>\n5.4071<\/td>\n2.8911<\/td>\n48.6964<\/td>\n51.3036<\/td>\n35.9304<\/td>\n10.2982<\/td>\n49.0054<\/td>\n4.7643<\/td>\n<\/tr>\n
      \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      Now I want to look at each row and remove any that are missing too many values. It’s okay to have a couple of missing data points in your dataset, but if you have too many it could cause your machine learning algorithm to be less accurate. I’ll use the Clean Missing Data<\/a> live task to remove any rows that are missing 2 or more data points.<\/div>\n
      \n
      \n
      % Remove missing data<\/span><\/span><\/div>\n<\/div>\n
      \n
      [fullData,missingIndices] = rmmissing(allTrainData,“MinNumMissing”<\/span>,2);<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      % Display results<\/span><\/span><\/div>\n<\/div>\n
      \n
      figure<\/span><\/div>\n<\/div>\n
      \n
      % Get locations of missing data<\/span><\/span><\/div>\n<\/div>\n
      \n
      indicesForPlot = ismissing(allTrainData.patient_age);<\/span><\/div>\n<\/div>\n
      \n
      mask = missingIndices & ~indicesForPlot;<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      % Plot cleaned data<\/span><\/span><\/div>\n<\/div>\n
      \n
      plot(find(~missingIndices),fullData.patient_age,“SeriesIndex”<\/span>,1,“LineWidth”<\/span>,1.5, …<\/span><\/span><\/div>\n<\/div>\n
      \n
      “DisplayName”<\/span>,“Cleaned data”<\/span>)<\/span><\/div>\n<\/div>\n
      \n
      hold on<\/span><\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      % Plot data in rows where other variables contain missing entries<\/span><\/span><\/div>\n<\/div>\n
      \n
      plot(find(mask),allTrainData.patient_age(mask),“x”<\/span>,“SeriesIndex”<\/span>,“none”<\/span>, …<\/span><\/span><\/div>\n<\/div>\n
      \n
      “DisplayName”<\/span>,“Removed by other variables”<\/span>)<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      % Plot removed missing entries<\/span><\/span><\/div>\n<\/div>\n
      \n
      x = repelem(find(indicesForPlot),3);<\/span><\/div>\n<\/div>\n
      \n
      y = repmat([ylim(gca) missing]’,nnz(indicesForPlot),1);<\/span><\/div>\n<\/div>\n
      \n
      plot(x,y,“Color”<\/span>,[145 145 145]\/255,“DisplayName”<\/span>,“Removed missing entries”<\/span>)<\/span><\/div>\n<\/div>\n
      \n
      title(“Number of removed missing entries: ” <\/span>+ nnz(indicesForPlot))<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      hold off<\/span><\/span><\/div>\n<\/div>\n
      \n
      legend<\/span><\/div>\n<\/div>\n
      \n
      ylabel(“patient_age”<\/span>,“Interpreter”<\/span>,“none”<\/span>)<\/span><\/div>\n
      \n
      \n
      <\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      clear indicesForPlot mask x y<\/span><\/span><\/div>\n<\/div>\n<\/div>\n

      Explore the Data<\/h2>\n
      Now that the data is cleaned up, you should spend some time exploring your data to understand how different variables may interact with each other or see if you can draw any meaningful conclusions from the data or figure out which variables may be more or less important when it comes to predicting time to diagnosis.<\/div>\n

      Univariate Analysis<\/h3>\n
      First, I want to separate the data into two datasets: one full of patients who were diagnosed in 90 days or less (the 1 or “True” values), and one full of patients who were not (the 0 or “False” values). This will allow me to explore the data patterns in each of these datasets and look for any meaningful differences.<\/div>\n
      \n
      \n
      allTrueIdx = fullData.DiagPeriodL90D == 1;<\/span><\/div>\n<\/div>\n
      \n
      allTrueData = fullData(allTrueIdx, :);<\/span><\/div>\n
      \n
      \n
      \n
      allTrueData = 7559\u00d780 table <\/span><\/div>\n
      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
      <\/th>\npatient_id<\/th>\npatient_race<\/th>\npayer_type<\/th>\npatient_state<\/th>\npatient_zip3<\/th>\npatient_age<\/th>\npatient_gender<\/th>\nbreast_cancer_diagnosis_code<\/th>\nbreast_cancer_diagnosis_desc<\/th>\nmetastatic_cancer_diagnosis_code<\/th>\nRegion<\/th>\nDivision<\/th>\npopulation<\/th>\ndensity<\/th>\nage_median<\/th>\nage_under_10<\/th>\nage_10_to_19<\/th>\nage_20s<\/th>\nage_30s<\/th>\nage_40s<\/th>\nage_50s<\/th>\nage_60s<\/th>\nage_70s<\/th>\nage_over_80<\/th>\nmale<\/th>\nfemale<\/th>\nmarried<\/th>\ndivorced<\/th>\nnever_married<\/th>\nwidowed<\/th>\n<\/tr>\n<\/thead>\n
      1<\/th>\n475714<\/td>\n<undefined><\/td>\nMEDICAID<\/td>\nCA<\/td>\n924<\/td>\n84<\/td>\nF<\/td>\nC50919<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0unspecified\u00a0female\u00a0breast<\/td>\nC7989<\/td>\nWest<\/td>\nPacific<\/td>\n3.1438e+04<\/td>\n1.1896e+03<\/td>\n30.6429<\/td>\n16.0143<\/td>\n15.5429<\/td>\n17.6143<\/td>\n14.0143<\/td>\n11.6143<\/td>\n11.5571<\/td>\n7.5714<\/td>\n4<\/td>\n2.1000<\/td>\n49.8571<\/td>\n50.1429<\/td>\n36.5714<\/td>\n11.8857<\/td>\n47.1143<\/td>\n4.4429<\/td>\n<\/tr>\n
      2<\/th>\n349367<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n928<\/td>\n62<\/td>\nF<\/td>\nC50411<\/td>\nMalig\u00a0neoplm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nPacific<\/td>\n3.9122e+04<\/td>\n2.2959e+03<\/td>\n38.2000<\/td>\n11.8788<\/td>\n13.3545<\/td>\n14.2303<\/td>\n13.4182<\/td>\n13.3333<\/td>\n14.0606<\/td>\n10.2485<\/td>\n5.9515<\/td>\n3.5030<\/td>\n49.8939<\/td>\n50.1061<\/td>\n50.2455<\/td>\n9.8273<\/td>\n35.2909<\/td>\n4.6515<\/td>\n<\/tr>\n
      3<\/th>\n138632<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nTX<\/td>\n760<\/td>\n43<\/td>\nF<\/td>\nC50112<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0central\u00a0portion\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nSouth<\/td>\nWest\u00a0South\u00a0Central<\/td>\n2.1997e+04<\/td>\n626.2367<\/td>\n37.9067<\/td>\n13.0283<\/td>\n14.4633<\/td>\n12.5317<\/td>\n13.5450<\/td>\n12.8600<\/td>\n12.7700<\/td>\n11.4267<\/td>\n6.5650<\/td>\n2.8117<\/td>\n50.1233<\/td>\n49.8767<\/td>\n55.7533<\/td>\n12.3300<\/td>\n27.1950<\/td>\n4.7100<\/td>\n<\/tr>\n
      4<\/th>\n914071<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n900<\/td>\n51<\/td>\nF<\/td>\nC50912<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC779<\/td>\nWest<\/td>\nPacific<\/td>\n3.6054e+04<\/td>\n5.2943e+03<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n11.3015<\/td>\n50.4569<\/td>\n4.7662<\/td>\n<\/tr>\n
      5<\/th>\n479368<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nIL<\/td>\n619<\/td>\n60<\/td>\nF<\/td>\nC50512<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0lower-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n3.4041e+03<\/td>\n25.7333<\/td>\n42.7900<\/td>\n11.9833<\/td>\n13.2567<\/td>\n9.5733<\/td>\n12.4000<\/td>\n11.8133<\/td>\n13.5767<\/td>\n14.0433<\/td>\n8.5267<\/td>\n4.8533<\/td>\n49.2833<\/td>\n50.7167<\/td>\n55.8867<\/td>\n12.6400<\/td>\n24.5267<\/td>\n6.9433<\/td>\n<\/tr>\n
      6<\/th>\n155485<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nIL<\/td>\n617<\/td>\n64<\/td>\nF<\/td>\nC50912<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n4.4353e+03<\/td>\n68.0019<\/td>\n41.3000<\/td>\n12.8358<\/td>\n13.6811<\/td>\n10.5245<\/td>\n11.9377<\/td>\n11.6585<\/td>\n13.5774<\/td>\n13.7434<\/td>\n7.6868<\/td>\n4.3415<\/td>\n49.3962<\/td>\n50.6038<\/td>\n57.8962<\/td>\n10.8981<\/td>\n24.9547<\/td>\n6.2472<\/td>\n<\/tr>\n
      7<\/th>\n266700<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nMI<\/td>\n480<\/td>\n58<\/td>\nF<\/td>\nC50812<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0ovrlp\u00a0sites\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC781<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n1.6938e+04<\/td>\n894.1681<\/td>\n42.9348<\/td>\n10.5116<\/td>\n11.8130<\/td>\n11.9217<\/td>\n12.4043<\/td>\n12.4304<\/td>\n15.2710<\/td>\n13.8826<\/td>\n7.8522<\/td>\n3.8971<\/td>\n50.0217<\/td>\n49.9783<\/td>\n50.9565<\/td>\n12.3145<\/td>\n30.8333<\/td>\n5.9014<\/td>\n<\/tr>\n
      8<\/th>\n880521<\/td>\nOther<\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n945<\/td>\n58<\/td>\nF<\/td>\nC50911<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nPacific<\/td>\n3.0154e+04<\/td>\n976.2892<\/td>\n42.1358<\/td>\n10.7531<\/td>\n12.7148<\/td>\n11.7259<\/td>\n13.1012<\/td>\n12.8173<\/td>\n13.3012<\/td>\n12.7716<\/td>\n8.4136<\/td>\n4.4086<\/td>\n49.7272<\/td>\n50.2728<\/td>\n53.0765<\/td>\n10.9123<\/td>\n30.5346<\/td>\n5.4667<\/td>\n<\/tr>\n
      9<\/th>\n971531<\/td>\nHispanic<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nIL<\/td>\n606<\/td>\n83<\/td>\nF<\/td>\nC50911<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n4.8671e+04<\/td>\n6.4314e+03<\/td>\n35.7554<\/td>\n10.4286<\/td>\n10.6518<\/td>\n18.3107<\/td>\n18.9036<\/td>\n11.9696<\/td>\n11.7268<\/td>\n9.6839<\/td>\n5.4071<\/td>\n2.8911<\/td>\n48.6964<\/td>\n51.3036<\/td>\n35.9304<\/td>\n10.2982<\/td>\n49.0054<\/td>\n4.7643<\/td>\n<\/tr>\n
      10<\/th>\n529840<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nMT<\/td>\n590<\/td>\n60<\/td>\nF<\/td>\nC50411<\/td>\nMalig\u00a0neoplm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nMountain<\/td>\n1.2208e+03<\/td>\n2.1597<\/td>\n46.7408<\/td>\n11.1521<\/td>\n11.1000<\/td>\n7.9183<\/td>\n10.3338<\/td>\n10.7577<\/td>\n15.4211<\/td>\n18.9042<\/td>\n9.4479<\/td>\n5<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/td>\n<\/tr>\n
      11<\/th>\n198037<\/td>\nWhite<\/td>\nMEDICAID<\/td>\nKY<\/td>\n402<\/td>\n45<\/td>\nF<\/td>\nC50312<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0lower-inner\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nSouth<\/td>\nEast\u00a0South\u00a0Central<\/td>\n2.2669e+04<\/td>\n1.1427e+03<\/td>\n37.4937<\/td>\n10.9688<\/td>\n13.6031<\/td>\n15.2281<\/td>\n14.9219<\/td>\n11.7219<\/td>\n12.1375<\/td>\n11.5188<\/td>\n6.3156<\/td>\n3.5625<\/td>\n48.8344<\/td>\n51.1656<\/td>\n39.7906<\/td>\n15.0312<\/td>\n39.2875<\/td>\n5.8906<\/td>\n<\/tr>\n
      12<\/th>\n791301<\/td>\n<undefined><\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nCA<\/td>\n958<\/td>\n58<\/td>\nF<\/td>\nC50112<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0central\u00a0portion\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nPacific<\/td>\n3.0687e+04<\/td>\n1.9179e+03<\/td>\n36.5517<\/td>\n11.6207<\/td>\n11.4655<\/td>\n16.1345<\/td>\n15.9655<\/td>\n12.5276<\/td>\n12.4793<\/td>\n11.0655<\/td>\n5.6034<\/td>\n3.1586<\/td>\n49.5138<\/td>\n50.4862<\/td>\n41.5345<\/td>\n13.7034<\/td>\n40.1793<\/td>\n4.5793<\/td>\n<\/tr>\n
      13<\/th>\n618259<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nOH<\/td>\n430<\/td>\n55<\/td>\nF<\/td>\nC50311<\/td>\nMalig\u00a0neoplm\u00a0of\u00a0lower-inner\u00a0quadrant\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n1.4386e+04<\/td>\n263.5774<\/td>\n40.6393<\/td>\n11.8852<\/td>\n14.2492<\/td>\n10.8426<\/td>\n11.5590<\/td>\n12.6984<\/td>\n13.8869<\/td>\n12.8131<\/td>\n8.2557<\/td>\n3.8148<\/td>\n49.7082<\/td>\n50.2918<\/td>\n53.7148<\/td>\n13.5279<\/td>\n25.8443<\/td>\n6.9180<\/td>\n<\/tr>\n
      14<\/th>\n393934<\/td>\nWhite<\/td>\n<undefined><\/td>\nCO<\/td>\n801<\/td>\n70<\/td>\nF<\/td>\nC50911<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC7951<\/td>\nWest<\/td>\nMountain<\/td>\n2.1243e+04<\/td>\n564.7743<\/td>\n42.4114<\/td>\n10.4086<\/td>\n14.4486<\/td>\n10.6314<\/td>\n11.8086<\/td>\n13.7086<\/td>\n15.9543<\/td>\n13.2914<\/td>\n6.9514<\/td>\n2.8057<\/td>\n50.7971<\/td>\n49.2029<\/td>\n60.1086<\/td>\n10.8800<\/td>\n25.5914<\/td>\n3.4286<\/td>\n<\/tr>\n
      \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n
      \n
      allFalseIdx = fullData.DiagPeriodL90D == 0;<\/span><\/div>\n<\/div>\n
      \n
      allFalseData = fullData(allFalseIdx, :);<\/span><\/div>\n
      \n
      \n
      \n
      allFalseData = 4598\u00d780 table <\/span><\/div>\n
      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
      <\/th>\npatient_id<\/th>\npatient_race<\/th>\npayer_type<\/th>\npatient_state<\/th>\npatient_zip3<\/th>\npatient_age<\/th>\npatient_gender<\/th>\nbreast_cancer_diagnosis_code<\/th>\nbreast_cancer_diagnosis_desc<\/th>\nmetastatic_cancer_diagnosis_code<\/th>\nRegion<\/th>\nDivision<\/th>\npopulation<\/th>\ndensity<\/th>\nage_median<\/th>\nage_under_10<\/th>\nage_10_to_19<\/th>\nage_20s<\/th>\nage_30s<\/th>\nage_40s<\/th>\nage_50s<\/th>\nage_60s<\/th>\nage_70s<\/th>\nage_over_80<\/th>\nmale<\/th>\nfemale<\/th>\nmarried<\/th>\ndivorced<\/th>\nnever_married<\/th>\nwidowed<\/th>\n<\/tr>\n<\/thead>\n
      1<\/th>\n617843<\/td>\nWhite<\/td>\nCOMMERCIAL<\/td>\nCA<\/td>\n926<\/td>\n45<\/td>\nF<\/td>\nC50212<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0upper-inner\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nWest<\/td>\nPacific<\/td>\n3.2795e+04<\/td>\n1.8962e+03<\/td>\n42.8714<\/td>\n10.0714<\/td>\n12.1357<\/td>\n12.5381<\/td>\n12.4643<\/td>\n12.6500<\/td>\n14.8476<\/td>\n12.2810<\/td>\n8.2167<\/td>\n4.7595<\/td>\n49.0667<\/td>\n50.9333<\/td>\n52.6048<\/td>\n11.6238<\/td>\n31.1429<\/td>\n4.6238<\/td>\n<\/tr>\n
      2<\/th>\n817482<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nID<\/td>\n836<\/td>\n55<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC773<\/td>\nWest<\/td>\nMountain<\/td>\n1.0886e+04<\/td>\n116.8860<\/td>\n43.4735<\/td>\n10.8240<\/td>\n13.9760<\/td>\n9.4920<\/td>\n10.3640<\/td>\n12.6000<\/td>\n14.9920<\/td>\n14.8360<\/td>\n9.4620<\/td>\n3.4660<\/td>\n52.3120<\/td>\n47.6880<\/td>\n57.8820<\/td>\n14.9640<\/td>\n21.7600<\/td>\n5.4060<\/td>\n<\/tr>\n
      3<\/th>\n111545<\/td>\nWhite<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nNY<\/td>\n141<\/td>\n66<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC7981<\/td>\nNortheast<\/td>\nMiddle\u00a0Atlantic<\/td>\n5.6438e+03<\/td>\n219.3629<\/td>\n45.1800<\/td>\n8.5114<\/td>\n14.8571<\/td>\n11.0886<\/td>\n9.7543<\/td>\n13.6143<\/td>\n13.3743<\/td>\n15.6857<\/td>\n9.4457<\/td>\n3.6457<\/td>\n50.9114<\/td>\n49.0914<\/td>\n51.3229<\/td>\n11.7600<\/td>\n30.8314<\/td>\n6.0914<\/td>\n<\/tr>\n
      4<\/th>\n875977<\/td>\n<undefined><\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nMI<\/td>\n488<\/td>\n67<\/td>\nF<\/td>\nC50412<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC799<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n8101<\/td>\n246.2810<\/td>\n40.2782<\/td>\n11.0456<\/td>\n14.7684<\/td>\n13.3848<\/td>\n11.4671<\/td>\n11.2203<\/td>\n14.8975<\/td>\n12.5899<\/td>\n7.1494<\/td>\n3.4709<\/td>\n51.3228<\/td>\n48.6772<\/td>\n49.0658<\/td>\n13.6051<\/td>\n31.8848<\/td>\n5.4392<\/td>\n<\/tr>\n
      5<\/th>\n343914<\/td>\n<undefined><\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nCA<\/td>\n900<\/td>\n66<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC7800<\/td>\nWest<\/td>\nPacific<\/td>\n3.6054e+04<\/td>\n5.2943e+03<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n11.3015<\/td>\n50.4569<\/td>\n4.7662<\/td>\n<\/tr>\n
      6<\/th>\n615208<\/td>\nOther<\/td>\nCOMMERCIAL<\/td>\nOR<\/td>\n975<\/td>\n62<\/td>\nF<\/td>\nC50411<\/td>\nMalig\u00a0neoplm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0right\u00a0female\u00a0breast<\/td>\nC786<\/td>\nWest<\/td>\nPacific<\/td>\n1.2836e+04<\/td>\n87.3667<\/td>\n48.9208<\/td>\n9.3458<\/td>\n9.4500<\/td>\n8.7833<\/td>\n11.9542<\/td>\n10.3458<\/td>\n12.6000<\/td>\n17.8833<\/td>\n13.8708<\/td>\n5.7542<\/td>\n50.4708<\/td>\n49.5292<\/td>\n53.7167<\/td>\n15.8583<\/td>\n23.1333<\/td>\n7.2708<\/td>\n<\/tr>\n
      7<\/th>\n279917<\/td>\nWhite<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nNY<\/td>\n142<\/td>\n75<\/td>\nF<\/td>\nC50912<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC7801<\/td>\nNortheast<\/td>\nMiddle\u00a0Atlantic<\/td>\n2.0195e+04<\/td>\n2.1920e+03<\/td>\n36.4690<\/td>\n10.2207<\/td>\n15.4345<\/td>\n17.8241<\/td>\n13.2483<\/td>\n10.2897<\/td>\n11.7345<\/td>\n11.5276<\/td>\n5.8966<\/td>\n3.8345<\/td>\n48.4414<\/td>\n51.5586<\/td>\n31.7517<\/td>\n12.9966<\/td>\n49.4724<\/td>\n5.7759<\/td>\n<\/tr>\n
      8<\/th>\n366792<\/td>\nAsian<\/td>\nCOMMERCIAL<\/td>\nMI<\/td>\n482<\/td>\n46<\/td>\nF<\/td>\nC50412<\/td>\nMalig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n2.2081e+04<\/td>\n1.6665e+03<\/td>\n36.5861<\/td>\n12.7778<\/td>\n12.8556<\/td>\n15.8083<\/td>\n13.2028<\/td>\n11.8889<\/td>\n12.4556<\/td>\n11.6000<\/td>\n5.9194<\/td>\n3.4833<\/td>\n48.5472<\/td>\n51.4528<\/td>\n26.2417<\/td>\n14.7028<\/td>\n52.5722<\/td>\n6.4611<\/td>\n<\/tr>\n
      9<\/th>\n643360<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nNY<\/td>\n120<\/td>\n52<\/td>\nF<\/td>\n1744<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0female\u00a0breast<\/td>\nC7800<\/td>\nNortheast<\/td>\nMiddle\u00a0Atlantic<\/td>\n5.1122e+03<\/td>\n103.9061<\/td>\n46.2954<\/td>\n9.0636<\/td>\n10.6182<\/td>\n11.3000<\/td>\n10.9576<\/td>\n11.0045<\/td>\n16.7348<\/td>\n15.3530<\/td>\n10.3818<\/td>\n4.5909<\/td>\n52.1348<\/td>\n47.8652<\/td>\n49.7773<\/td>\n13.7470<\/td>\n29.3818<\/td>\n7.0924<\/td>\n<\/tr>\n
      10<\/th>\n487817<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nTX<\/td>\n773<\/td>\n57<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC773<\/td>\nSouth<\/td>\nWest\u00a0South\u00a0Central<\/td>\n2.4751e+04<\/td>\n352.2268<\/td>\n41.3712<\/td>\n11.9302<\/td>\n12.9868<\/td>\n10.9962<\/td>\n11.1623<\/td>\n13.1075<\/td>\n13.0226<\/td>\n13.0660<\/td>\n9.5774<\/td>\n4.1585<\/td>\n49.3547<\/td>\n50.6453<\/td>\n52.9943<\/td>\n13.3415<\/td>\n25.0943<\/td>\n8.5792<\/td>\n<\/tr>\n
      11<\/th>\n345047<\/td>\n<undefined><\/td>\nCOMMERCIAL<\/td>\nTX<\/td>\n751<\/td>\n78<\/td>\nF<\/td>\nC50912<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast<\/td>\nC773<\/td>\nSouth<\/td>\nWest\u00a0South\u00a0Central<\/td>\n1.6981e+04<\/td>\n271.9135<\/td>\n38.5392<\/td>\n13.2529<\/td>\n15.1843<\/td>\n11.8118<\/td>\n11.7980<\/td>\n12.7176<\/td>\n14.0510<\/td>\n11.8373<\/td>\n6.3667<\/td>\n2.9627<\/td>\n49.8667<\/td>\n50.1333<\/td>\n52.8333<\/td>\n13.5725<\/td>\n28.0196<\/td>\n5.5725<\/td>\n<\/tr>\n
      12<\/th>\n907418<\/td>\nWhite<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nIN<\/td>\n460<\/td>\n50<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC7951<\/td>\nMidwest<\/td>\nEast\u00a0North\u00a0Central<\/td>\n1.3549e+04<\/td>\n256.8795<\/td>\n40.2864<\/td>\n13.3023<\/td>\n13.2045<\/td>\n10.9000<\/td>\n14.2909<\/td>\n12.6364<\/td>\n13.2909<\/td>\n11.8773<\/td>\n6.3886<\/td>\n4.0886<\/td>\n50.8545<\/td>\n49.1455<\/td>\n54.1114<\/td>\n12.4795<\/td>\n27.4068<\/td>\n5.9955<\/td>\n<\/tr>\n
      13<\/th>\n908851<\/td>\nWhite<\/td>\nMEDICARE\u00a0ADVANTAGE<\/td>\nFL<\/td>\n339<\/td>\n82<\/td>\nF<\/td>\n1749<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified<\/td>\nC7800<\/td>\nSouth<\/td>\nSouth\u00a0Atlantic<\/td>\n1.8007e+04<\/td>\n479.7347<\/td>\n50.9592<\/td>\n7.9449<\/td>\n9.9816<\/td>\n10.4449<\/td>\n9.7082<\/td>\n9.3714<\/td>\n13.1020<\/td>\n16.6653<\/td>\n16.0143<\/td>\n6.7714<\/td>\n49.0306<\/td>\n50.9694<\/td>\n52.1673<\/td>\n13.8000<\/td>\n25.2510<\/td>\n8.7939<\/td>\n<\/tr>\n
      14<\/th>\n785337<\/td>\nBlack<\/td>\nCOMMERCIAL<\/td>\nVA<\/td>\n234<\/td>\n54<\/td>\nF<\/td>\n1741<\/td>\nMalignant\u00a0neoplasm\u00a0of\u00a0central\u00a0portion\u00a0of\u00a0female\u00a0breast<\/td>\nC7951<\/td>\nSouth<\/td>\nSouth\u00a0Atlantic<\/td>\n1.3242e+04<\/td>\n299.2533<\/td>\n44.9310<\/td>\n9.1227<\/td>\n10.6818<\/td>\n13.8364<\/td>\n11.3295<\/td>\n9.5705<\/td>\n14.7477<\/td>\n14.3523<\/td>\n12.4568<\/td>\n3.8909<\/td>\n50.5205<\/td>\n49.4795<\/td>\n50.1318<\/td>\n13.8023<\/td>\n29.3364<\/td>\n6.7386<\/td>\n<\/tr>\n
      \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      Now we can use the Create Plot live task<\/a> to plot histograms of the different variables in each dataset. In the plot below, blue bars represent data from the folks who were diagnosed in a timely manner, and the red bars represent data from the folks who were not.<\/div>\n
      \n
      \n
      figure<\/span><\/div>\n<\/div>\n
      \n
      % Create histogram of selected data<\/span><\/span><\/div>\n<\/div>\n
      \n
      histogram(allTrueData.health_uninsured,“NumBins”<\/span>,40,“DisplayName”<\/span>,“health_uninsured”<\/span>);<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      hold on<\/span><\/span><\/div>\n<\/div>\n
      \n
      % Create histogram of selected data<\/span><\/span><\/div>\n<\/div>\n
      \n
      histogram(allFalseData.health_uninsured,“NumBins”<\/span>,40,“DisplayName”<\/span>,“health_uninsured”<\/span>);<\/span><\/div>\n<\/div>\n
      \n
      hold off<\/span><\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      legend<\/span><\/div>\n
      \n
      \n
      <\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      Take some time to explore these visualizations on your own, as I can only show one at a time in this blog. It is worth noting that we have less False data than True data, so the red bars will almost always be lower than the blue bars. If there are red bars that are higher or if the shapes are different, that may indicate a relationship between a variable and time to diagnosis.<\/div>\n
      I didn’t see many significant differences in shape, though I did notice that for the ‘health_uninsured’ histograms the red vars are fairly high in the higher numbers, indicating that there may<\/span> be a correlation between populations with high rates of being unisured and time to diagnosis.<\/div>\n

      Bivariate and Multivariate Analysis<\/h3>\n
      You can break the data down further and plot two (or more!) variables against each other to see if you can find any patterns. In the plot below, for example, we can see the percentage of the population that is unisured and the state the patient is in, broken down by whether or not the patient was diagnosed within 90 days. Again, blue values indicate that the patient was, and red values indicate that the patient was not.<\/div>\n
      \n
      \n
      figure<\/span><\/div>\n<\/div>\n
      \n
      % Create scatter of selected data<\/span><\/span><\/div>\n<\/div>\n
      \n
      scatter(allTrueData,“patient_state”<\/span>,“health_uninsured”<\/span>,“DisplayName”<\/span>,“health_uninsured”<\/span>);<\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      hold on<\/span><\/span><\/div>\n<\/div>\n
      \n
      % Create scatter of selected data<\/span><\/span><\/div>\n<\/div>\n
      \n
      scatter(allFalseData,“patient_state”<\/span>,“health_uninsured”<\/span>,“DisplayName”<\/span>,“health_uninsured”<\/span>);<\/span><\/div>\n<\/div>\n
      \n
      hold off<\/span><\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      legend<\/span><\/div>\n
      \n
      \n
      <\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      We can see that in some states, such as GA, OK, or TX, the the red values come from populations that are typically higher in terms of being uninsured. This could indcate that in some states, coming from a zip code with a high population of uninsured folks (or being uninsured yourself) means you are more likely to receive delays in your diagnosis.<\/div>\n

      Statistical Analysis<\/h3>\n
      You can also create meaningful deductions by calculating various statistics from your data. For example, I want to calculate the skewness<\/span><\/a>, or level of asymmetry, of each of my variables. A negative value indicates the data is left skewed when plotted, and a positive value indicates the data is right skewed when plotted, with a 0 meaning the data is evenly distributed.<\/div>\n
      \n
      \n
      statsTrue = varfun(@skewness, allTrueData, “InputVariables”<\/span>, @isnumeric);<\/span><\/div>\n<\/div>\n
      \n
      statsFalse = varfun(@skewness, allFalseData, “InputVariables”<\/span>, @isnumeric);<\/span><\/div>\n<\/div>\n<\/div>\n
      Now I want to see if any of the variables have a significant difference in their skewness, as differences in the data distributions between patients who were diagnosed in a timely manner vs patients who were not could indicate an underlying relationship between those variables and time to diagnosis.<\/div>\n
      \n
      \n
      statsDiffs = abs(statsTrue{:, :} – statsFalse{:, :});<\/span><\/div>\n<\/div>\n
      \n
      statsTrue.Properties.VariableNames(statsDiffs > 0.2)<\/span><\/div>\n
      \n
      \n
      \n
      \n
      ans = <\/span>1\u00d74 cell<\/span><\/div>\n<\/div>\n
      \n
      ‘skewness_density”skewness_age_over_80”skewness_rent_burden”skewness_race_native’<\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      If we investigate the four variables that are returned, we can see that population density, the percentage of folks above 80 in your zip code, the median rent burden of your zip code, and the percentage of residents who reported their race as American Indian or Alaska Native<\/span> in your zip code may have a relationship with time to diagnosis.<\/div>\n

      Feature Engineering<\/h2>\n
      When it comes to machine learning, you don’t have to use all of the data as it is presented to you. Feature Engineering<\/a> is the process of deciding what data you want to use, creating new data based on the provided data, and transforming the data to be in whatever format or range is suitable for your workflow. You can do this manually, and some of the exploration we just did should influence decisions you make if you want to play around with including or excluding different variables.<\/div>\n
      For this blog, I’ll use the gencfeatures<\/span><\/a> function to automate this process. I want to use 90 features, which is 10 more than we currently have in our dataset, and it will go through and create a set of 90 meaningful features based on our processed dataset. It may keep some data as-is, but will often standardize numeric variables and create new variables by manipulating the provided data.<\/div>\n
      \n
      \n
      [T, augTrainData] = gencfeatures(fullData, “DiagPeriodL90D”<\/span>, 90)<\/span><\/div>\n
      \n
      \n
      \n
      Warning: Table variable names were truncated to the length namelengthmax.<\/div>\n
      <\/div>\n<\/div>\n<\/div>\n
      \n
      \n
      T = <\/span><\/div>\n
      \n

      FeatureTransformer with properties:<\/p>\n

      Type: ‘classification’
      \nTargetLearner: ‘linear’
      \nNumEngineeredFeatures: 89
      \nNumOriginalFeatures: 1
      \nTotalNumFeatures: 90<\/p>\n<\/div>\n<\/div>\n<\/div>\n

      \n
      \n
      augTrainData = 12157\u00d791 table <\/span><\/div>\n
      \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
      <\/th>\nmetastatic_cancer_diagnosis_code<\/th>\nzsc(woe2(breast_cancer_diagnosis_code))<\/th>\nzsc(woe2(breast_cancer_diagnosis_desc))<\/th>\nzsc(woe2(metastatic_cancer_diagnosis_code))<\/th>\nzsc(woe2(patient_state))<\/th>\nzsc(patient_age.\/Ozone)<\/th>\nzsc(patient_age.\/commute_time)<\/th>\nzsc(kmc51)<\/th>\neb28(education_less_highschool)<\/th>\nzsc(income_household_35_to_50.\/income_household_75_to_100)<\/th>\nzsc(kmc12)<\/th>\neb11(patient_age)<\/th>\nq28(income_household_under_5)<\/th>\nzsc(rent_burden-education_less_highschool)<\/th>\nq11(patient_age)<\/th>\nzsc(sig(family_dual_income))<\/th>\nzsc(sig(patient_age))<\/th>\nzsc(sin(PM25))<\/th>\nzsc(cos(rent_median))<\/th>\nzsc(sin(patient_zip3))<\/th>\nzsc(health_uninsured.\/PM25)<\/th>\nzsc(cos(population))<\/th>\nzsc(cos(education_bachelors))<\/th>\nzsc(sin(hispanic))<\/th>\nq28(density)<\/th>\neb28(education_highschool)<\/th>\nzsc(income_household_75_to_100.*rent_burden)<\/th>\nq28(unemployment_rate)<\/th>\nq28(patient_zip3)<\/th>\nzsc(patient_id.*hispanic)<\/th>\n<\/tr>\n<\/thead>\n
      1<\/th>\nC7989<\/td>\n0.5390<\/td>\n0.5390<\/td>\n-0.4180<\/td>\n0.3566<\/td>\n0.2943<\/td>\n1.2850<\/td>\n-0.9221<\/td>\n28<\/td>\n0.4148<\/td>\n0.8408<\/td>\n11<\/td>\n17<\/td>\n-2.6485<\/td>\n11<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.4182<\/td>\n-1.3204<\/td>\n0.4400<\/td>\n0.1652<\/td>\n-1.3368<\/td>\n-0.8396<\/td>\n-0.9848<\/td>\n19<\/td>\n16<\/td>\n1.3688<\/td>\n25<\/td>\n25<\/td>\n1.8601<\/td>\n<\/tr>\n
      2<\/th>\nC773<\/td>\n0.5403<\/td>\n0.5403<\/td>\n0.6847<\/td>\n0.3566<\/td>\n-0.0953<\/td>\n-0.2533<\/td>\n1.1450<\/td>\n14<\/td>\n-1.0911<\/td>\n-0.6892<\/td>\n7<\/td>\n9<\/td>\n0.2093<\/td>\n7<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.5817<\/td>\n0.4439<\/td>\n-1.4209<\/td>\n-0.5852<\/td>\n-1.2593<\/td>\n0.1155<\/td>\n0.3359<\/td>\n24<\/td>\n8<\/td>\n0.6588<\/td>\n11<\/td>\n26<\/td>\n0.2666<\/td>\n<\/tr>\n
      3<\/th>\nC773<\/td>\n0.4472<\/td>\n0.4472<\/td>\n0.6847<\/td>\n-0.4817<\/td>\n-1.1723<\/td>\n-1.2683<\/td>\n-0.5463<\/td>\n9<\/td>\n-0.1595<\/td>\n0.1389<\/td>\n3<\/td>\n10<\/td>\n-0.1364<\/td>\n2<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.8441<\/td>\n-0.5097<\/td>\n-0.4515<\/td>\n1.2506<\/td>\n1.0719<\/td>\n0.8394<\/td>\n0.7034<\/td>\n14<\/td>\n14<\/td>\n0.0868<\/td>\n7<\/td>\n19<\/td>\n-0.6458<\/td>\n<\/tr>\n
      4<\/th>\nC773<\/td>\n0.4804<\/td>\n0.4804<\/td>\n0.6847<\/td>\n0.3566<\/td>\n-1.1790<\/td>\n-0.8614<\/td>\n2.4460<\/td>\n2<\/td>\n-1.4278<\/td>\n-2.6472<\/td>\n3<\/td>\n19<\/td>\n1.2015<\/td>\n2<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.5895<\/td>\n0.0984<\/td>\n0.9142<\/td>\n-0.9343<\/td>\n-1.3164<\/td>\n-0.5676<\/td>\n-1.2673<\/td>\n23<\/td>\n1<\/td>\n-0.8003<\/td>\n13<\/td>\n26<\/td>\n0.0137<\/td>\n<\/tr>\n
      5<\/th>\nC773<\/td>\n-1.8389<\/td>\n-1.8389<\/td>\n0.6847<\/td>\n-0.1188<\/td>\n-0.4668<\/td>\n-0.1359<\/td>\n-0.1677<\/td>\n10<\/td>\n1.1998<\/td>\n0.5623<\/td>\n5<\/td>\n11<\/td>\n-0.6774<\/td>\n5<\/td>\n0.0164<\/td>\n0.0329<\/td>\n-1.9107<\/td>\n-1.1374<\/td>\n0.3927<\/td>\n2.8054<\/td>\n-1.0779<\/td>\n0.0894<\/td>\n0.9926<\/td>\n5<\/td>\n16<\/td>\n-0.8990<\/td>\n5<\/td>\n21<\/td>\n0.0631<\/td>\n<\/tr>\n
      6<\/th>\nC7981<\/td>\n-1.8389<\/td>\n-1.8389<\/td>\n-3.0232<\/td>\n-0.7796<\/td>\n0.3970<\/td>\n0.6970<\/td>\n-0.5463<\/td>\n5<\/td>\n-0.4166<\/td>\n0.1389<\/td>\n8<\/td>\n1<\/td>\n-0.1713<\/td>\n9<\/td>\n0.0164<\/td>\n0.0329<\/td>\n-0.8052<\/td>\n-1.4920<\/td>\n0.4399<\/td>\n-0.6668<\/td>\n0.1846<\/td>\n-1.5704<\/td>\n0.8916<\/td>\n8<\/td>\n23<\/td>\n-0.1033<\/td>\n3<\/td>\n3<\/td>\n-0.8538<\/td>\n<\/tr>\n
      7<\/th>\nC779<\/td>\n0.5150<\/td>\n0.5150<\/td>\n-1.0339<\/td>\n0.3566<\/td>\n-0.7176<\/td>\n-0.8180<\/td>\n1.2575<\/td>\n26<\/td>\n0.1398<\/td>\n-0.9325<\/td>\n4<\/td>\n28<\/td>\n-0.8526<\/td>\n4<\/td>\n0.0164<\/td>\n0.0329<\/td>\n-2.1568<\/td>\n0.2754<\/td>\n1.3443<\/td>\n-0.4261<\/td>\n0.5523<\/td>\n0.3214<\/td>\n1.4403<\/td>\n27<\/td>\n6<\/td>\n-0.0186<\/td>\n26<\/td>\n23<\/td>\n2.7136<\/td>\n<\/tr>\n
      8<\/th>\nC773<\/td>\n0.5605<\/td>\n05605<\/td>\n0.6847<\/td>\n-0.8787<\/td>\n0.2552<\/td>\n0.4365<\/td>\n-0.9221<\/td>\n8<\/td>\n1.2589<\/td>\n0.8408<\/td>\n6<\/td>\n12<\/td>\n-1.0383<\/td>\n7<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.6462<\/td>\n0.3778<\/td>\n-0.2288<\/td>\n0.0129<\/td>\n0.3019<\/td>\n1.0857<\/td>\n0.5697<\/td>\n1<\/td>\n26<\/td>\n-1.3912<\/td>\n17<\/td>\n17<\/td>\n-0.7649<\/td>\n<\/tr>\n
      9<\/th>\nC773<\/td>\n0.5150<\/td>\n0.5150<\/td>\n0.6847<\/td>\n-0.8787<\/td>\n0.5303<\/td>\n0.7654<\/td>\n-0.5463<\/td>\n1<\/td>\n-0.3771<\/td>\n0.1389<\/td>\n7<\/td>\n15<\/td>\n0.0934<\/td>\n8<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.6494<\/td>\n1.3120<\/td>\n1.2738<\/td>\n-0.8822<\/td>\n1.1952<\/td>\n1.2474<\/td>\n0.1205<\/td>\n3<\/td>\n23<\/td>\n-0.1783<\/td>\n2<\/td>\n17<\/td>\n-0.8367<\/td>\n<\/tr>\n
      10<\/th>\nC799<\/td>\n0.5892<\/td>\n0.5892<\/td>\n-0.7083<\/td>\n0.4285<\/td>\n0.6062<\/td>\n0.4731<\/td>\n-0.1677<\/td>\n5<\/td>\n0.3661<\/td>\n0.5623<\/td>\n8<\/td>\n4<\/td>\n-0.1511<\/td>\n9<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.8468<\/td>\n0.7382<\/td>\n-1.3160<\/td>\n-0.3069<\/td>\n-0.5051<\/td>\n-1.0839<\/td>\n-1.0734<\/td>\n9<\/td>\n22<\/td>\n-0.5364<\/td>\n11<\/td>\n13<\/td>\n-0.5800<\/td>\n<\/tr>\n
      11<\/th>\nC7800<\/td>\n-1.8389<\/td>\n-1.8389<\/td>\n-0.4676<\/td>\n0.3566<\/td>\n0.2790<\/td>\n-0.0623<\/td>\n2.5856<\/td>\n26<\/td>\n0.1398<\/td>\n-3.1658<\/td>\n8<\/td>\n28<\/td>\n-0.8526<\/td>\n9<\/td>\n0.0164<\/td>\n0.0329<\/td>\n-2.1568<\/td>\n0.2754<\/td>\n1.3443<\/td>\n-0.4261<\/td>\n0.5523<\/td>\n0.3214<\/td>\n1.4403<\/td>\n27<\/td>\n6<\/td>\n-0.0186<\/td>\n26<\/td>\n23<\/td>\n0.4736<\/td>\n<\/tr>\n
      12<\/th>\nC781<\/td>\n0.5758<\/td>\n0.5758<\/td>\n-1.7502<\/td>\n0.4285<\/td>\n-0.0598<\/td>\n-0.2406<\/td>\n-0.5463<\/td>\n4<\/td>\n-0.4701<\/td>\n0.1389<\/td>\n6<\/td>\n8<\/td>\n0.1148<\/td>\n6<\/td>\n0.0164<\/td>\n0.0329<\/td>\n0.8290<\/td>\n-1.2051<\/td>\n0.8002<\/td>\n-0.9475<\/td>\n0.0945<\/td>\n1.1447<\/td>\n0.5680<\/td>\n16<\/td>\n14<\/td>\n-0.0036<\/td>\n16<\/td>\n12<\/td>\n-0.8148<\/td>\n<\/tr>\n
      13<\/th>\nC773<\/td>\n0.5668<\/td>\n0.5668<\/td>\n0.6847<\/td>\n0.3566<\/td>\n0.2459<\/td>\n-0.8014<\/td>\n1.2575<\/td>\n6<\/td>\n-1.3361<\/td>\n-0.9325<\/td>\n6<\/td>\n5<\/td>\n0.6748<\/td>\n6<\/td>\n0.0164<\/td>\n0.0329<\/td>\n-0.6779<\/td>\n0.9431<\/td>\n0.7497<\/td>\n-0.9270<\/td>\n0.9452<\/td>\n-0.4288<\/td>\n0.1632<\/td>\n17<\/td>\n4<\/td>\n-0.4941<\/td>\n13<\/td>\n27<\/td>\n0.7836<\/td>\n<\/tr>\n
      14<\/th>\nC786<\/td>\n0.5403<\/td>\n0.5403<\/td>\n-0.0208<\/td>\n0.4096<\/td>\n0.4423<\/td>\n0.4481<\/td>\n-1.0650<\/td>\n7<\/td>\n1.5972<\/td>\n0.0484<\/td>\n7<\/td>\n22<\/td>\n1.3062<\/td>\n7<\/td>\n0.0164<\/td>\n0.0329<\/td>\n-1.4776<\/td>\n0.6570<\/td>\n1.1964<\/td>\n0.8045<\/td>\n1.4144<\/td>\n0.4459<\/td>\n1.3659<\/td>\n4<\/td>\n15<\/td>\n0.5212<\/td>\n22<\/td>\n28<\/td>\n-0.4435<\/td>\n<\/tr>\n
      \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      To better understand the generated features, you can use the describe<\/span><\/a> function of the returned FeatureTransformer<\/span><\/a> object, ‘T’.<\/div>\n
      \n
      \n
      describe(T)<\/span><\/div>\n
      \n
      \n
      Type<\/strong> IsOriginal<\/strong> InputVariables<\/strong> Transformations<\/strong>
      \n___________<\/strong> __________<\/strong> _____________________________________________________<\/strong> ________________________________________________________________________<\/strong>metastatic_cancer_diagnosis_code <\/strong> Categorical true metastatic_cancer_diagnosis_code
      \nzsc(woe2(breast_cancer_diagnosis_code)) <\/strong> Numeric false breast_cancer_diagnosis_code Weight of Evidence (positive class = 1)
      \nStandardization with z-score (mean = -0.046637, std = 1.5098)
      \nzsc(woe2(breast_cancer_diagnosis_desc)) <\/strong> Numeric false breast_cancer_diagnosis_desc Weight of Evidence (positive class = 1)
      \nStandardization with z-score (mean = -0.046637, std = 1.5098)
      \nzsc(woe2(metastatic_cancer_diagnosis_code)) <\/strong> Numeric false metastatic_cancer_diagnosis_code Weight of Evidence (positive class = 1)
      \nStandardization with z-score (mean = 0.0067098, std = 0.28786)
      \nzsc(woe2(patient_state)) <\/strong> Numeric false patient_state Weight of Evidence (positive class = 1)
      \nStandardization with z-score (mean = 0.0060064, std = 0.23323)
      \nzsc(patient_age.\/Ozone) <\/strong> Numeric false patient_age, Ozone patient_age .\/ Ozone
      \nStandardization with z-score (mean = 1.5005, std = 0.36544)
      \nzsc(patient_age.\/commute_time) <\/strong> Numeric false patient_age, commute_time patient_age .\/ commute_time
      \nStandardization with z-score (mean = 2.1895, std = 0.64638)
      \nzsc(kmc51) <\/strong> Numeric false all valid numeric variables Centroid encoding (component #51) (kmeans clustering with k = 10)
      \nStandardization with z-score (mean = 5.9447, std = 0.1673)
      \neb28(education_less_highschool) <\/strong> Categorical false education_less_highschool Equal-width binning (number of bins = 28)
      \nzsc(income_household_35_to_50.\/income_household_75_to_100)<\/strong> Numeric false income_household_35_to_50, income_household_75_to_100 income_household_35_to_50 .\/ income_household_75_to_100
      \nStandardization with z-score (mean = 0.93234, std = 0.2685)
      \nzsc(kmc12) <\/strong> Numeric false all valid numeric variables Centroid encoding (component #12) (kmeans clustering with k = 10)
      \nStandardization with z-score (mean = 13.4409, std = 0.15797)
      \neb11(patient_age) <\/strong> Categorical false patient_age Equal-width binning (number of bins = 11)
      \nq28(income_household_under_5) <\/strong> Categorical false income_household_under_5 Equiprobable binning (number of bins = 28)
      \nzsc(rent_burden-education_less_highschool) <\/strong> Numeric false rent_burden, education_less_highschool rent_burden – education_less_highschool
      \nStandardization with z-score (mean = 19.3265, std = 5.7168)
      \nq11(patient_age) <\/strong> Categorical false patient_age Equiprobable binning (number of bins = 11)
      \nzsc(sig(family_dual_income)) <\/strong> Numeric false family_dual_income sigmoid( )
      \nStandardization with z-score (mean = 1, std = 4.2283e-11)
      \nzsc(sig(patient_age)) <\/strong> Numeric false patient_age sigmoid( )
      \nStandardization with z-score (mean = 1, std = 4.0863e-10)
      \nzsc(sin(PM25)) <\/strong> Numeric false PM25 sin( )
      \nStandardization with z-score (mean = 0.42558, std = 0.65419)
      \nzsc(cos(rent_median)) <\/strong> Numeric false rent_median cos( )
      \nStandardization with z-score (mean = 0.046444, std = 0.68827)
      \nzsc(sin(patient_zip3)) <\/strong> Numeric false patient_zip3 sin( )
      \nStandardization with z-score (mean = 0.054487, std = 0.70171)
      \nzsc(health_uninsured.\/PM25) <\/strong> Numeric false health_uninsured, PM25 health_uninsured .\/ PM25
      \nStandardization with z-score (mean = 1.1917, std = 0.6234)
      \nzsc(cos(population)) <\/strong> Numeric false population cos( )
      \nStandardization with z-score (mean = -0.03209, std = 0.71354)
      \nzsc(cos(education_bachelors)) <\/strong> Numeric false education_bachelors cos( )
      \nStandardization with z-score (mean = 0.096871, std = 0.68966)
      \nzsc(sin(hispanic)) <\/strong> Numeric false hispanic sin( )
      \nStandardization with z-score (mean = 0.017785, std = 0.6817)
      \nq28(density) <\/strong> Categorical false density Equiprobable binning (number of bins = 28)
      \neb28(education_highschool) <\/strong> Categorical false education_highschool Equal-width binning (number of bins = 28)
      \nzsc(income_household_75_to_100.*rent_burden) <\/strong> Numeric false income_household_75_to_100, rent_burden income_household_75_to_100 .* rent_burden
      \nStandardization with z-score (mean = 392.7502, std = 61.6458)
      \nq28(unemployment_rate) <\/strong> Categorical false unemployment_rate Equiprobable binning (number of bins = 28)
      \nq28(patient_zip3) <\/strong> Categorical false patient_zip3 Equiprobable binning (number of bins = 28)
      \nzsc(patient_id.*hispanic) <\/strong> Numeric false patient_id, hispanic patient_id .* hispanic
      \nStandardization with z-score (mean = 10169065.2502, std = 11587944.1233)
      \nzsc(home_value.*race_other) <\/strong> Numeric false home_value, race_other home_value .* race_other
      \nStandardization with z-score (mean = 2725364.3718, std = 4298818.8992)
      \nzsc(patient_age.*income_household_20_to_25) <\/strong> Numeric false patient_age, income_household_20_to_25 patient_age .* income_household_20_to_25
      \nStandardization with z-score (mean = 241.7171, std = 97.8001)
      \nq25(farmer) <\/strong> Categorical false farmer Equiprobable binning (number of bins = 25)
      \nq27(race_native) <\/strong> Categorical false race_native Equiprobable binning (number of bins = 27)
      \neb28(age_median) <\/strong> Categorical false age_median Equal-width binning (number of bins = 28)
      \nq28(never_married) <\/strong> Categorical false never_married Equiprobable binning (number of bins = 28)
      \nzsc(cos(patient_age)) <\/strong> Numeric false patient_age cos( )
      \nStandardization with z-score (mean = 0.021113, std = 0.71469)
      \nzsc(sin(race_black)) <\/strong> Numeric false race_black sin( )
      \nStandardization with z-score (mean = 0.16517, std = 0.70668)
      \nzsc(tanh(age_50s)) <\/strong> Numeric false age_50s tanh( )
      \nStandardization with z-score (mean = 1, std = 8.9224e-09)
      \nzsc(male+female) <\/strong> Numeric false male, female male + female
      \nStandardization with z-score (mean = 100.0001, std = 0.000436)
      \nq28(female) <\/strong> Categorical false female Equiprobable binning (number of bins = 28)
      \neb28(male) <\/strong> Categorical false male Equal-width binning (number of bins = 28)
      \nzsc(sin(age_median)) <\/strong> Numeric false age_median sin( )
      \nStandardization with z-score (mean = -0.1365, std = 0.71613)
      \nq28(home_ownership) <\/strong> Categorical false home_ownership Equiprobable binning (number of bins = 28)
      \nzsc(age_over_80.\/income_household_20_to_25) <\/strong> Numeric false age_over_80, income_household_20_to_25 age_over_80 .\/ income_household_20_to_25
      \nStandardization with z-score (mean = 1.0866, std = 0.51568)
      \nzsc(cos(education_highschool)) <\/strong> Numeric false education_highschool cos( )
      \nStandardization with z-score (mean = -0.019221, std = 0.71994)
      \nzsc(cos(race_black)) <\/strong> Numeric false race_black cos( )
      \nStandardization with z-score (mean = -0.020693, std = 0.68773)
      \nq28(self_employed) <\/strong> Categorical false self_employed Equiprobable binning (number of bins = 28)
      \nzsc(cos(age_median)) <\/strong> Numeric false age_median cos( )
      \nStandardization with z-score (mean = -0.029038, std = 0.68394)
      \nq50(patient_id) <\/strong> Categorical false patient_id Equiprobable binning (number of bins = 50)
      \nzsc(sin(race_asian)) <\/strong> Numeric false race_asian sin( )
      \nStandardization with z-score (mean = 0.28421, std = 0.64235)
      \nq28(education_stem_degree) <\/strong> Categorical false education_stem_degree Equiprobable binning (number of bins = 28)
      \nzsc(cos(age_20s)) <\/strong> Numeric false age_20s cos( )
      \nStandardization with z-score (mean = 0.10518, std = 0.69162)
      \neb23(N02) <\/strong> Categorical false N02 Equal-width binning (number of bins = 23)
      \nq28(rent_burden) <\/strong> Categorical false rent_burden Equiprobable binning (number of bins = 28)
      \nzsc(race_asian.*veteran) <\/strong> Numeric false race_asian, veteran race_asian .* veteran
      \nStandardization with z-score (mean = 28.4889, std = 30.7)
      \nzsc(sin(income_household_35_to_50)) <\/strong> Numeric false income_household_35_to_50 sin( )
      \nStandardization with z-score (mean = 0.03083, std = 0.68752)
      \nzsc(cos(patient_zip3)) <\/strong> Numeric false patient_zip3 cos( )
      \nStandardization with z-score (mean = -0.06867, std = 0.7071)
      \neb28(rent_burden) <\/strong> Categorical false rent_burden Equal-width binning (number of bins = 28)
      \nzsc(sig(rent_burden)) <\/strong> Numeric false rent_burden sigmoid( )
      \nStandardization with z-score (mean = 1, std = 3.571e-10)
      \nq28(age_over_80) <\/strong> Categorical false age_over_80 Equiprobable binning (number of bins = 28)
      \nq28(family_dual_income) <\/strong> Categorical false family_dual_income Equiprobable binning (number of bins = 28)
      \nq28(family_size) <\/strong> Categorical false family_size Equiprobable binning (number of bins = 28)
      \nzsc(age_over_80.\/income_household_5_to_10) <\/strong> Numeric false age_over_80, income_household_5_to_10 age_over_80 .\/ income_household_5_to_10
      \nStandardization with z-score (mean = 2.0422, std = 1.3415)
      \neb28(age_10_to_19) <\/strong> Categorical false age_10_to_19 Equal-width binning (number of bins = 28)
      \nq28(income_individual_median) <\/strong> Categorical false income_individual_median Equiprobable binning (number of bins = 28)
      \nzsc(age_over_80.\/unemployment_rate) <\/strong> Numeric false age_over_80, unemployment_rate age_over_80 .\/ unemployment_rate
      \nStandardization with z-score (mean = 0.74942, std = 0.37691)
      \nzsc(cos(income_household_50_to_75)) <\/strong> Numeric false income_household_50_to_75 cos( )
      \nStandardization with z-score (mean = -0.012865, std = 0.69717)
      \neb25(race_pacific) <\/strong> Categorical false race_pacific Equal-width binning (number of bins = 25)
      \nzsc(sin(patient_id)) <\/strong> Numeric false patient_id sin( )
      \nStandardization with z-score (mean = -0.0018454, std = 0.70739)
      \nzsc(race_native.\/race_multiple) <\/strong> Numeric false race_native, race_multiple race_native .\/ race_multiple
      \nStandardization with z-score (mean = 0.14079, std = 0.41944)
      \neb28(income_household_25_to_35) <\/strong> Categorical false income_household_25_to_35 Equal-width binning (number of bins = 28)
      \nzsc(age_50s-income_household_75_to_100) <\/strong> Numeric false age_50s, income_household_75_to_100 age_50s – income_household_75_to_100
      \nStandardization with z-score (mean = 0.77657, std = 2.1264)
      \nzsc(cos(age_60s)) <\/strong> Numeric false age_60s cos( )
      \nStandardization with z-score (mean = 0.05337, std = 0.75178)
      \nq28(income_household_35_to_50) <\/strong> Categorical false income_household_35_to_50 Equiprobable binning (number of bins = 28)
      \neb21(race_black) <\/strong> Categorical false race_black Equal-width binning (number of bins = 21)
      \nzsc(sin(income_individual_median)) <\/strong> Numeric false income_individual_median sin( )
      \nStandardization with z-score (mean = 0.045145, std = 0.69873)
      \nq28(age_50s) <\/strong> Categorical false age_50s Equiprobable binning (number of bins = 28)
      \nq28(race_white) <\/strong> Categorical false race_white Equiprobable binning (number of bins = 28)
      \nq28(age_under_10) <\/strong> Categorical false age_under_10 Equiprobable binning (number of bins = 28)
      \nq28(disabled) <\/strong> Categorical false disabled Equiprobable binning (number of bins = 28)
      \nzsc(patient_age.\/income_household_100_to_150) <\/strong> Numeric false patient_age, income_household_100_to_150 patient_age .\/ income_household_100_to_150
      \nStandardization with z-score (mean = 3.9266, std = 1.314)
      \nq28(income_household_75_to_100) <\/strong> Categorical false income_household_75_to_100 Equiprobable binning (number of bins = 28)
      \nzsc(sin(N02)) <\/strong> Numeric false N02 sin( )
      \nStandardization with z-score (mean = 0.039533, std = 0.70149)
      \neb28(family_size) <\/strong> Categorical false family_size Equal-width binning (number of bins = 28)
      \nq28(limited_english) <\/strong> Categorical false limited_english Equiprobable binning (number of bins = 28)
      \nq28(income_household_100_to_150) <\/strong> Categorical false income_household_100_to_150 Equiprobable binning (number of bins = 28)
      \nzsc(farmer.*race_black) <\/strong> Numeric false farmer, race_black farmer .* race_black
      \nStandardization with z-score (mean = 10.7649, std = 26.8957)
      \nzsc(home_value.*race_pacific) <\/strong> Numeric false home_value, race_pacific home_value .* race_pacific
      \nStandardization with z-score (mean = 59826.8413, std = 128896.4218)
      \nzsc(education_graduate.*health_uninsured) <\/strong> Numeric false education_graduate, health_uninsured education_graduate .* health_uninsured
      \nStandardization with z-score (mean = 97.7642, std = 54.0304)<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

      Split the Data<\/h2>\n
      The last step before you can train a machine learning model is to split your data into a training and testing set. We’ll use the training data to fit the model, and the testing set to evaluate how well the model performs on new data before we use it to make a submission. Here I split the data into 80% training and 20% testing.<\/div>\n
      \n
      \n
      numRows = height(augTrainData);<\/span><\/div>\n<\/div>\n
      \n
      [trainInd, ~, testInd] = dividerand(numRows, .8, 0, .2); <\/span><\/div>\n<\/div>\n
      \n
      <\/div>\n<\/div>\n
      \n
      trainingData = augTrainData(trainInd, :);<\/span><\/div>\n<\/div>\n
      \n
      testingData = augTrainData(testInd, :);<\/span><\/div>\n<\/div>\n<\/div>\n

      Train a Machine Learning Model<\/h2>\n
      In this example, I’ll create a binary decision tree using the fitctree<\/span><\/a> function and set ‘Optimize Hyperparameters’<\/a> to ‘auto’, which will attempt to minimize the error of our algorithm by choosing the best value for the ‘MinLeafSize’ parameter. It visualizes the results of adjusting this value, as can be seen below.<\/div>\n
      \n
      \n
      classificationTree = fitctree(trainingData, “DiagPeriodL90D”<\/span>, …<\/span><\/span><\/div>\n<\/div>\n
      \n
      OptimizeHyperparameters=‘auto’<\/span>);<\/span><\/div>\n
      \n
      \n
      |======================================================================================|
      \n| Iter | Eval | Objective | Objective | BestSoFar | BestSoFar | MinLeafSize |
      \n| | result | | runtime | (observed) | (estim.) | |
      \n|======================================================================================|
      \n| 1 | Best | 0.18764 | 1.4699 | 0.18764 | 0.18764 | 1676 |
      \n| 2 | Accept | 0.18764 | 0.87349 | 0.18764 | 0.18764 | 162 |
      \n| 3 | Accept | 0.20923 | 1.005 | 0.18764 | 0.19426 | 36 |
      \n| 4 | Accept | 0.29395 | 1.6132 | 0.18764 | 0.18764 | 3 |
      \n| 5 | Accept | 0.18764 | 0.6073 | 0.18764 | 0.1876 | 491 |
      \n| 6 | Accept | 0.38012 | 0.21492 | 0.18764 | 0.24104 | 4858 |
      \n| 7 | Accept | 0.18764 | 0.60759 | 0.18764 | 0.18764 | 330 |
      \n| 8 | Accept | 0.18764 | 0.36986 | 0.18764 | 0.18763 | 1033 |
      \n| 9 | Accept | 0.19227 | 1.0609 | 0.18764 | 0.18762 | 80 |
      \n| 10 | Accept | 0.24409 | 1.4868 | 0.18764 | 0.18761 | 13 |
      \n| 11 | Accept | 0.18764 | 0.3479 | 0.18764 | 0.18568 | 1363 |
      \n| 12 | Accept | 0.18764 | 0.70426 | 0.18764 | 0.1861 | 231 |
      \n| 13 | Accept | 0.18764 | 0.48941 | 0.18764 | 0.18678 | 698 |
      \n| 14 | Accept | 0.29519 | 2.1238 | 0.18764 | 0.18671 | 1 |
      \n| 15 | Accept | 0.18764 | 0.35153 | 0.18764 | 0.18736 | 1438 |
      \n| 16 | Accept | 0.18764 | 0.86203 | 0.18764 | 0.18735 | 119 |
      \n| 17 | Accept | 0.18764 | 0.41595 | 0.18764 | 0.18734 | 849 |
      \n| 18 | Accept | 0.18764 | 0.31486 | 0.18764 | 0.18737 | 1527 |
      \n| 19 | Accept | 0.18764 | 0.60161 | 0.18764 | 0.18738 | 404 |
      \n| 20 | Accept | 0.18764 | 0.45615 | 0.18764 | 0.18738 | 589 |
      \n|======================================================================================|
      \n| Iter | Eval | Objective | Objective | BestSoFar | BestSoFar | MinLeafSize |
      \n| | result | | runtime | (observed) | (estim.) | |
      \n|======================================================================================|
      \n| 21 | Accept | 0.18764 | 0.30864 | 0.18764 | 0.18745 | 1515 |
      \n| 22 | Accept | 0.18764 | 0.71981 | 0.18764 | 0.18745 | 138 |
      \n| 23 | Accept | 0.18764 | 0.62974 | 0.18764 | 0.18745 | 278 |
      \n| 24 | Accept | 0.18764 | 0.27013 | 0.18764 | 0.18749 | 1511 |
      \n| 25 | Accept | 0.18764 | 0.62894 | 0.18764 | 0.18749 | 196 |
      \n| 26 | Accept | 0.18764 | 0.40254 | 0.18764 | 0.18749 | 811 |
      \n| 27 | Accept | 0.30239 | 0.19617 | 0.18764 | 0.18741 | 2944 |
      \n| 28 | Accept | 0.18764 | 0.27176 | 0.18764 | 0.18741 | 1170 |
      \n| 29 | Accept | 0.18764 | 0.37273 | 0.18764 | 0.18747 | 1576 |
      \n| 30 | Accept | 0.18764 | 0.45381 | 0.18764 | 0.18747 | 945 |__________________________________________________________
      \nOptimization completed.
      \nMaxObjectiveEvaluations of 30 reached.
      \nTotal function evaluations: 30
      \nTotal elapsed time: 50.9097 seconds
      \nTotal objective function evaluation time: 20.2308Best observed feasible point:
      \nMinLeafSize<\/strong>
      \n___________<\/strong>1676Observed objective function value = 0.18764
      \nEstimated objective function value = 0.18815
      \nFunction evaluation time = 1.4699Best estimated feasible point (according to models):
      \nMinLeafSize<\/strong>
      \n___________<\/strong>1527Estimated objective function value = 0.18747
      \nEstimated function evaluation time = 0.36237<\/div>\n<\/div>\n
      \n
      <\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n
      \n
      <\/div>\n
      <\/div>\n
      <\/div>\n
      <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
      I used a binary tree as my starting point, but it’s important to test out different types of algorithms to see what works best with your data! Check out the Classification Learner app documentation<\/a> and this short video<\/a> to learn how to train several machine learning models quickly and iteratively!<\/div>\n

      Test Your Model<\/h2>\n
      There are many ways to evaluate the performance of a machine learning model, so in this blog I’ll show how to do so by computing validation accuracy and using testing data.<\/div>\n

      Validation Accuracy<\/h3>\n
      Cross-validation<\/a> is one method of evaluating a model, and at a high level is done by:<\/div>\n
        \n
      1. Setting aside a subset of the training data, known as validation data<\/li>\n
      2. Using the rest of the training data to fit the model<\/li>\n
      3. Testing how well the model performs on the validation data<\/li>\n<\/ol>\n
        You can use the crossval<\/span><\/a> function to do this:<\/div>\n
        \n
        \n
        % Perform cross-validation<\/span><\/span><\/div>\n<\/div>\n
        \n
        partitionedModel = crossval(classificationTree, ‘KFold’<\/span>, 5);<\/span><\/div>\n<\/div>\n<\/div>\n
        Then, extract the misclassification rate, and subtract it from 1 to get the model’s accuracy. The closer to 1 this value is, the more accurate our model is.<\/div>\n
        \n
        \n
        % Compute validation accuracy<\/span><\/span><\/div>\n<\/div>\n
        \n
        validationAccuracy = 1 – kfoldLoss(partitionedModel, LossFun=‘ClassifError’<\/span>)<\/span><\/div>\n
        \n
        validationAccuracy = 0.8124<\/div>\n<\/div>\n<\/div>\n<\/div>\n

        Testing Data<\/h3>\n
        In this section, we’ll use the ‘testingData’ dataset we created earlier<\/a>. Similar to what we did with the validation data, we can use the loss<\/span><\/a> function to compute the misclassification rate when you use the classification tree on the testing data, and subtract it from 1 to get a measure of accuracy.<\/div>\n
        \n
        \n
        testAccuracy = 1 – loss(classificationTree, testingData, “DiagPeriodL90D”<\/span>,…<\/span><\/span><\/div>\n<\/div>\n
        \n
        LossFun=‘classiferror’<\/span>)<\/span><\/div>\n
        \n
        testAccuracy = 0.8048<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        I also want to compare the predictions that the model makes to the actual outputs, so let’s remove the ‘DiagPeriodL90D’ variable from our testing data<\/div>\n
        \n
        \n
        testActual = testingData.DiagPeriodL90D;<\/span><\/div>\n<\/div>\n
        \n
        testingData = removevars(testingData, “DiagPeriodL90D”<\/span>);<\/span><\/div>\n<\/div>\n<\/div>\n
        Now, use the model to make predictions on the testing set<\/div>\n
        \n
        \n
        [testPreds, scores, ~, ~] = predict(classificationTree, testingData);<\/span><\/div>\n<\/div>\n<\/div>\n
        And use the confusionchart<\/span><\/a> function to compare the predicted outputs to the actual outputs, to see how often they match or don’t.<\/div>\n
        \n
        \n
        confusionchart(testActual, testPreds)<\/span><\/div>\n
        \n
        \n
        <\/div>\n
        <\/div>\n
        <\/div>\n
        <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        This shows that it almost always predicts 1s correctly, or when the patient is diagnosed within 90 days, but it’s almost a 50\/50 chance that this model will predict the 0s correctly.<\/div>\n
        We can also use the test data and predictions to visualize receiver operating characteristic (ROC)<\/a> metrics. The ROC curve shows the true positive rate (TPR) versus the false positive rate (FPR) for different thresholds of classification scores. The “Model Operating Point” shows the false positive rate and true positive rate of the model.<\/div>\n
        \n
        \n
        rocObj = rocmetrics(testActual, scores, classificationTree.ClassNames);<\/span><\/div>\n<\/div>\n
        \n
        plot(rocObj)<\/span><\/div>\n
        \n
        \n
        <\/div>\n
        <\/div>\n
        <\/div>\n
        <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        Here we can see that the classifier correctly assigns about 90-95% of the 1 class observations to 1 (TPR), but incorrectly assigns about 40% of the 0 class observations as 1 (FPR). This is similar to what we observed with the confusion chart.<\/div>\n
        You can also extract the area under the curve (AUC) value, which is a measure of the overall quality of the classifier. The AUC values are in the range 0<\/span> to 1<\/span>, and larger AUC values indicate better classifier performance.<\/div>\n
        \n
        \n
        rocObj.AUC<\/span><\/div>\n<\/div>\n<\/div>\n
        The AUC is pretty high, but shows that there is definitely room for improvement. To learn more about ROC metrics, check out this documentation page<\/a> that explains it in more detail.<\/div>\n

        Create Submission<\/h2>\n
        Once you have a model that performs well on the validation and testing data, it’s time to create a submission for the datathon! As a reminder, you will upload this file to Kaggle to be scored on the leaderboard.<\/div>\n
        First, import the ‘Test’ dataset:<\/div>\n
        \n
        \n
        testDataFilename = ‘Test.csv’<\/span>;<\/span><\/div>\n<\/div>\n
        \n
        allTestData = readtable(fullfile(dataFolder, testDataFilename))<\/span><\/div>\n
        \n
        \n
        \n
        allTestData = 3999\u00d783 table <\/span><\/div>\n
        \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
        <\/th>\npatient_id<\/th>\npatient_race<\/th>\npayer_type<\/th>\npatient_state<\/th>\npatient_zip3<\/th>\npatient_age<\/th>\npatient_gender<\/th>\nbmi<\/th>\nbreast_cancer_diagnosis_code<\/th>\nbreast_cancer_diagnosis_desc<\/th>\nmetastatic_cancer_diagnosis_code<\/th>\nmetastatic_first_novel_treatment<\/th>\nmetastatic_first_novel_treatment_type<\/th>\nRegion<\/th>\nDivision<\/th>\npopulation<\/th>\ndensity<\/th>\nage_median<\/th>\nage_under_10<\/th>\nage_10_to_19<\/th>\nage_20s<\/th>\nage_30s<\/th>\nage_40s<\/th>\nage_50s<\/th>\nage_60s<\/th>\nage_70s<\/th>\nage_over_80<\/th>\nmale<\/th>\nfemale<\/th>\nmarried<\/th>\n<\/tr>\n<\/thead>\n
        1<\/th>\n573710<\/td>\n‘White’<\/td>\n‘MEDICAID’<\/td>\n‘IN’<\/td>\n467<\/td>\n54<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50412’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n5.4414e+03<\/td>\n85.6210<\/td>\n40.8803<\/td>\n12.7323<\/td>\n14.0887<\/td>\n10.6597<\/td>\n11.6258<\/td>\n11.2081<\/td>\n15.6194<\/td>\n12.3226<\/td>\n8.4097<\/td>\n3.3435<\/td>\n49.1548<\/td>\n50.8452<\/td>\n55.1758<\/td>\n<\/tr>\n
        2<\/th>\n593679<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘FL’<\/td>\n337<\/td>\n52<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50912’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C787’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘South’<\/td>\n‘South\u00a0Atlantic’<\/td>\n1.9614e+04<\/td>\n1.5551e+03<\/td>\n49.1077<\/td>\n8.0692<\/td>\n8.5872<\/td>\n10.6846<\/td>\n11.3026<\/td>\n10.9718<\/td>\n15.8231<\/td>\n15.9026<\/td>\n11.8282<\/td>\n6.8154<\/td>\n49.6590<\/td>\n50.3410<\/td>\n44.8000<\/td>\n<\/tr>\n
        3<\/th>\n184532<\/td>\n‘Hispanic’<\/td>\n‘MEDICAID’<\/td>\n‘CA’<\/td>\n917<\/td>\n61<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50911’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0right\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n4.3030e+04<\/td>\n2.0486e+03<\/td>\n38.8522<\/td>\n11.3065<\/td>\n12.8978<\/td>\n14.1217<\/td>\n13.5326<\/td>\n13.1609<\/td>\n13.3783<\/td>\n11.4739<\/td>\n6.3804<\/td>\n3.7370<\/td>\n49.0522<\/td>\n50.9478<\/td>\n48.5043<\/td>\n<\/tr>\n
        4<\/th>\n184532<\/td>\n‘Hispanic’<\/td>\n‘MEDICARE\u00a0ADVANTAGE’<\/td>\n‘CA’<\/td>\n917<\/td>\n61<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50912’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C779’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n4.3030e+04<\/td>\n2.0486e+03<\/td>\n38.8522<\/td>\n11.3065<\/td>\n12.8978<\/td>\n14.1217<\/td>\n13.5326<\/td>\n13.1609<\/td>\n13.3783<\/td>\n11.4739<\/td>\n6.3804<\/td>\n3.7370<\/td>\n49.0522<\/td>\n50.9478<\/td>\n48.5043<\/td>\n<\/tr>\n
        5<\/th>\n447383<\/td>\n‘Black’<\/td>\n”<\/td>\n‘CA’<\/td>\n917<\/td>\n64<\/td>\n‘F’<\/td>\n23<\/td>\n‘C50412’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C779’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.6054e+04<\/td>\n5.2943e+03<\/td>\n36.6538<\/td>\n9.7615<\/td>\n11.2677<\/td>\n17.2338<\/td>\n17.4415<\/td>\n13.0908<\/td>\n12.3046<\/td>\n9.4077<\/td>\n5.6738<\/td>\n3.8246<\/td>\n50.5108<\/td>\n49.4892<\/td>\n33.4785<\/td>\n<\/tr>\n
        6<\/th>\n281312<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘MI’<\/td>\n483<\/td>\n64<\/td>\n‘F’<\/td>\n24<\/td>\n‘1748’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0other\u00a0specified\u00a0sites\u00a0of\u00a0female\u00a0breast’<\/td>\n‘C7800’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n2.0151e+04<\/td>\n724.9353<\/td>\n42.0784<\/td>\n11.0392<\/td>\n13.0098<\/td>\n11.6431<\/td>\n11.8882<\/td>\n13.0647<\/td>\n15.1098<\/td>\n12.8686<\/td>\n7.4000<\/td>\n3.9588<\/td>\n49.2922<\/td>\n50.7078<\/td>\n54.0137<\/td>\n<\/tr>\n
        7<\/th>\n492714<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘TX’<\/td>\n761<\/td>\n91<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50912’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘South’<\/td>\n‘West\u00a0South\u00a0Central’<\/td>\n2.9482e+04<\/td>\n1.3355e+03<\/td>\n33.6278<\/td>\n13.1611<\/td>\n15.3444<\/td>\n16.7250<\/td>\n15.2167<\/td>\n12.5361<\/td>\n11.4139<\/td>\n8.8583<\/td>\n4.4167<\/td>\n2.3250<\/td>\n47.6694<\/td>\n52.3306<\/td>\n43.4639<\/td>\n<\/tr>\n
        8<\/th>\n378266<\/td>\n‘White’<\/td>\n‘MEDICARE\u00a0ADVANTAGE’<\/td>\n‘IN’<\/td>\n473<\/td>\n79<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50212’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0upper-inner\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n5.2774e+03<\/td>\n296.8542<\/td>\n42.0763<\/td>\n11.0220<\/td>\n13.9932<\/td>\n12.1288<\/td>\n10.4949<\/td>\n12.3237<\/td>\n13.4797<\/td>\n13.8864<\/td>\n7.6407<\/td>\n5.0271<\/td>\n48.5627<\/td>\n51.4373<\/td>\n50.6559<\/td>\n<\/tr>\n
        9<\/th>\n291550<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘AZ’<\/td>\n852<\/td>\n50<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50919’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0unspecified\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘West’<\/td>\n‘Mountain’<\/td>\n3.5899e+04<\/td>\n1.1664e+03<\/td>\n41.8273<\/td>\n10.8364<\/td>\n12.3045<\/td>\n12.7114<\/td>\n12.7545<\/td>\n11.8909<\/td>\n13.0341<\/td>\n12.7659<\/td>\n9.1523<\/td>\n4.5614<\/td>\n49.7568<\/td>\n50.2432<\/td>\n51.5750<\/td>\n<\/tr>\n
        10<\/th>\n612272<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘CA’<\/td>\n902<\/td>\n47<\/td>\n‘F’<\/td>\n24<\/td>\n‘C50412’<\/td>\n‘Malig\u00a0neoplasm\u00a0of\u00a0upper-outer\u00a0quadrant\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C7801’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘West’<\/td>\n‘Pacific’<\/td>\n3.5350e+04<\/td>\n3.5588e+03<\/td>\n38.7486<\/td>\n11.0686<\/td>\n13.8657<\/td>\n13.6371<\/td>\n13.7886<\/td>\n13.7000<\/td>\n13.0686<\/td>\n10.7143<\/td>\n6.4571<\/td>\n3.6943<\/td>\n49.2714<\/td>\n50.7286<\/td>\n45.4657<\/td>\n<\/tr>\n
        11<\/th>\n240105<\/td>\n‘White’<\/td>\n‘MEDICAID’<\/td>\n‘CO’<\/td>\n802<\/td>\n56<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘C50919’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unsp\u00a0site\u00a0of\u00a0unspecified\u00a0female\u00a0breast’<\/td>\n‘C7931’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘West’<\/td>\n‘Mountain’<\/td>\n2.5754e+04<\/td>\n2.4639e+03<\/td>\n35.8175<\/td>\n10.1600<\/td>\n10.0575<\/td>\n18.0375<\/td>\n19.6900<\/td>\n13.9675<\/td>\n10.9375<\/td>\n8.9850<\/td>\n5.3875<\/td>\n2.7900<\/td>\n50.2375<\/td>\n49.7625<\/td>\n41.6875<\/td>\n<\/tr>\n
        12<\/th>\n277939<\/td>\n‘White’<\/td>\n‘MEDICAID’<\/td>\n‘KY’<\/td>\n401<\/td>\n44<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘1749’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified’<\/td>\n‘C7931’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘South’<\/td>\n‘East\u00a0South\u00a0Central’<\/td>\n4.9004e+03<\/td>\n64.2871<\/td>\n42.1097<\/td>\n11.2903<\/td>\n11.8742<\/td>\n12.5065<\/td>\n11.3323<\/td>\n11.7258<\/td>\n15.5484<\/td>\n13.4419<\/td>\n8.6645<\/td>\n3.6194<\/td>\n51.4742<\/td>\n48.5258<\/td>\n47.8613<\/td>\n<\/tr>\n
        13<\/th>\n504153<\/td>\n”<\/td>\n‘COMMERCIAL’<\/td>\n‘IL’<\/td>\n600<\/td>\n52<\/td>\n‘F’<\/td>\nNaN<\/td>\n‘1749’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0breast\u00a0(female),\u00a0unspecified’<\/td>\n‘C7931’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n2.5744e+04<\/td>\n981.7631<\/td>\n41.7625<\/td>\n11.7846<\/td>\n13.8677<\/td>\n10.5738<\/td>\n11.3246<\/td>\n12.5923<\/td>\n15.0154<\/td>\n13.0277<\/td>\n7.8185<\/td>\n3.9908<\/td>\n49.9282<\/td>\n50.0708<\/td>\n57.2108<\/td>\n<\/tr>\n
        14<\/th>\n287269<\/td>\n‘Asian’<\/td>\n‘COMMERCIAL’<\/td>\n‘IL’<\/td>\n606<\/td>\n58<\/td>\n‘F’<\/td>\n23<\/td>\n‘C50912’<\/td>\n‘Malignant\u00a0neoplasm\u00a0of\u00a0unspecified\u00a0site\u00a0of\u00a0left\u00a0female\u00a0breast’<\/td>\n‘C773’<\/td>\nNaN<\/td>\nNaN<\/td>\n‘Midwest’<\/td>\n‘East\u00a0North\u00a0Central’<\/td>\n4.8671e+04<\/td>\n6.4314e+03<\/td>\n35.7554<\/td>\n10.4286<\/td>\n10.6518<\/td>\n18.3107<\/td>\n18.9036<\/td>\n11.9696<\/td>\n11.7268<\/td>\n9.6839<\/td>\n5.4071<\/td>\n2.8911<\/td>\n48.6964<\/td>\n51.3036<\/td>\n35.9304<\/td>\n<\/tr>\n
        \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
        <\/div>\n
        <\/div>\n
        <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        Then we need to process this dataset in the same way that we did the training data. In this section, I use code instead of the live tasks for simplicity.<\/div>\n
        \n
        \n
        % replace cell arrays with categoricals<\/span><\/span><\/div>\n<\/div>\n
        \n
        varTypes = varfun(@class, allTestData, OutputFormat=“cell”<\/span>);<\/span><\/div>\n<\/div>\n
        \n
        catIdx = strcmp(varTypes, “cell”<\/span>);<\/span><\/div>\n<\/div>\n
        \n
        varNames = allTestData.Properties.VariableNames;<\/span><\/div>\n<\/div>\n
        \n
        catVarNames = varNames(catIdx);<\/span><\/div>\n<\/div>\n
        \n
        <\/div>\n<\/div>\n
        \n
        for <\/span>catNameIdx = 1:length(catVarNames)<\/span><\/div>\n<\/div>\n
        \n
        allTestData.(catVarNames{catNameIdx}) = categorical(allTestData.(catVarNames{catNameIdx}));<\/span><\/div>\n<\/div>\n
        \n
        end<\/span><\/span><\/div>\n<\/div>\n
        \n
        <\/div>\n<\/div>\n
        \n
        % remove variables with too many missing data points<\/span><\/span><\/div>\n<\/div>\n
        \n
        fullTestData = removevars(allTestData, [“bmi”<\/span>, “metastatic_first_novel_treatment”<\/span>, “metastatic_first_novel_treatment_type”<\/span>]);<\/span><\/div>\n<\/div>\n
        \n
        <\/div>\n<\/div>\n
        <\/div>\n
        \n
        <\/div>\n<\/div>\n<\/div>\n
        We also need to use the transform<\/span><\/a> function to create the same features as we created using gencfeatures<\/span> for the training data.<\/div>\n
        \n
        \n
        augTestData = transform(T, fullTestData);<\/span><\/div>\n<\/div>\n<\/div>\n
        Now that the data is in the format our machine learning model expects it to be in, use the predict<\/span><\/a> function to make predictions, and create a table to contain the patient IDs and corresponding predictions.<\/div>\n
        \n
        \n
        submissionPreds = predict(classificationTree, augTestData);<\/span><\/div>\n<\/div>\n
        \n
        submissionTable = table(fullTestData.patient_id, submissionPreds, VariableNames=[“patient_id”<\/span>, “DiagPeriodL90D”<\/span>])<\/span><\/div>\n
        \n
        \n
        \n
        submissionTable = 3780\u00d72 table <\/span><\/div>\n
        \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
        <\/th>\npatient_id<\/th>\nDiagPeriodL90D<\/th>\n<\/tr>\n<\/thead>\n
        1<\/th>\n573710<\/td>\n1<\/td>\n<\/tr>\n
        2<\/th>\n593679<\/td>\n1<\/td>\n<\/tr>\n
        3<\/th>\n184532<\/td>\n1<\/td>\n<\/tr>\n
        4<\/th>\n447383<\/td>\n1<\/td>\n<\/tr>\n
        5<\/th>\n687972<\/td>\n1<\/td>\n<\/tr>\n
        6<\/th>\n281312<\/td>\n0<\/td>\n<\/tr>\n
        7<\/th>\n492714<\/td>\n1<\/td>\n<\/tr>\n
        8<\/th>\n378266<\/td>\n1<\/td>\n<\/tr>\n
        9<\/th>\n291550<\/td>\n1<\/td>\n<\/tr>\n
        10<\/th>\n612272<\/td>\n1<\/td>\n<\/tr>\n
        11<\/th>\n240105<\/td>\n1<\/td>\n<\/tr>\n
        12<\/th>\n277939<\/td>\n0<\/td>\n<\/tr>\n
        13<\/th>\n504153<\/td>\n0<\/td>\n<\/tr>\n
        14<\/th>\n287269<\/td>\n1<\/td>\n<\/tr>\n
        \u22ee<\/th>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n
        <\/div>\n
        <\/div>\n
        <\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
        Last, export your predictions to a .CSV file, then upload to Kaggle for scoring.<\/div>\n
        \n
        \n
        writetable(submissionTable, “Predictions.csv”<\/span>);<\/span><\/div>\n<\/div>\n<\/div>\n
        And that’s it! Thank you for following along with this tutorial, and best of luck to all participants. If you have any questions about this tutorial or MATLAB, reach out to us at studentcompetitions@mathworks.com<\/span><\/a> or by tagging gracewoolson<\/a> in the forum. Keep your eye out for our upcoming WiDS Workshop on January 31st, where we will walk through this tutorial and answer any questions you have along the way!<\/div>\n<\/div>\n